Statistical analysis and selection criteria

Uncertainty Assignment Based on Number of Available Data 

As indicated in the Nuclear Energy Agency reviews, the assignment of an uncertainty is quite subjective. Typically, this is exacerbated by the fact that the majority of studies only quote uncertainties based on numerical treatment of the experimental data. They generally do not account for any kind of systematic error that might be involved in the experimental procedure used. Ekberg et al. (2000) included all relevant systematic errors involved in their potentiometric measurements for the determination of the hydrolysis constants of thorium(lV) and presented both the uncertainties derived as a result of the numerical treatment of the data and the total uncertainty where the systematic errors were also included. Their data showed that the total uncertainty was about a factor of 4-10 larger than that derived in the numerical treatment. 

The equation utilised to determine the total uncertainty is (Ekberg et al., 2000) 

Hydrolysis of Metal Ions, First Edition. Paul L. Brown and Christian Ekberg. 

For example, in the experiments of Ekberg et al. (2000), the systematic errors would have included those related to the measurement of pH, ionic strength, titrant concentration and volume addition, temperature, etc. Each Sj must be related to the X value and be expressed in the same units (Brown, Curti and Grambow, 2005). As indicated, the estimated total uncertainty is significantly larger than that measured from the numerical treatment. The total uncertainty is a measure of the reliability and reproducibility of the stability constant, whereas the standard deviation derived in the numerical treatment is a measure of the precision of the experiment only (Brown, Curti and Grambow, 2005). On the basis of the difference between the total uncertainty and the numerical standard deviation, the uncertainties accepted in the present review have been increased substantially from those quoted in the majority of the original studies. 

---

The article that follows is a controversial one. It reaches a controversial conclusion - that much of the therapeutic benefit of antidepressant medications actually derives from placebo responding. The article reaches this conclusion by utilizing a controversial statistical approach - meta-analysis. And it employs meta-analysis controversially - by meta-analysing studies that are very heterogeneous in subject selection criteria, treatments employed, and statistical methods used. Nonetheless, we have chosen to publish the article. We have done so because a number of the colleagues who originally reviewed the manuscript believed it had considerable merit, even while they recognized the clearly contentious conclusions it... 

The case-control method does have its problems. Characterization of past exposure experiences by both cases and controls is sometimes based on memory. Individuals will be able to recall their occupations, but may not be able to pinpoint specific chemicals. They almost certainly can not specify the quantity of exposure (chemical concentrations), although they may perhaps recall the total time they were exposed. Study interpretation is sometimes complicated by the possibility that cases and controls may differ in their recollections of past exposures if this is the case (and it s not easy to tell if it is), the study may not meet the criteria expected for a truly controlled situation. In some cases recollection can be checked by turning to documentary records of those exposures, but past record keeping on most occupational exposures was typically irregular and incomplete. Other problems exist in selecting appropriate controls, and there are sometimes serious technical difficulties associated with statistical analysis of study results. 

A number of statistical analyses of the literature have been carried out in an attempt to estimate the extent of polymorphism. A search of the Cambridge Structural Database on the keywords polymorph , form , modification and phase indicates that about 3.5% of the -350 000 entries fall into this category. Approximately 25 % of the entries are either solvates or hydrates. At the other end of the spectrum, Byrn has reported that of the >150 compounds submitted for crystal form screening and analysis to SSCI, Inc. 85 % exhibit more than one crystal form, 37% are solvates and 31 % are hydrates [40]. Other studies based on different selection criteria reveal results falling somewhere between these two extremes [41]. For instance, Griesser and Burger have collected information on about 600 polymorphic forms and solvates (including hydrates) pharmaceutical compounds that are solid at 25 °C [41c],... 

The four data sets in this problem were selected by Anscombe (1973) to have the same values for their slopes (= 0.50), their intercepts (= 3.00), the sums ofthe squares of their residuals S(y-ycaic)2 (= 1.53), their standard deviations ay (= 1.11), crai ( = 0.118), and crao (= 1.12), as well as their correlation coefficients r (=0.816). By all these criteria, then, they fit the same equation of a straight line equally well. However, visual examination of the graphs, or of the residual plots, yields a quite different answer only set 1 reasonably fits a straight line. In other words, the statistical analysis in terms of summary statistics does not address the validity of the assumed model, and can produce results regardless of whether or not the model is appropriate. In the present case, the first data set reasonably fits the assumed model of a linear dependence, the second set should be fitted to a parabola instead of to a line, while the third and fourth sets both contain an intentional outlier . Direct observation of the graphs before the analysis, and/or of the residuals afterwards, can often help us reject clearly inappropriate models. 

Statistical analysis demonstrates which input parameters are important contributors to output parameters [10]. IMP statistical software (SAS, Cary, NC) can be used to analyze data and generate separate analyses for each response variable of interest [61]. A difference of 3 SDs often is selected as the criterion for signihcance corresponding to a 95% conhdence level [61], and the test acceptance criteria range typically are set as the mean plus/minus 3 SDs (industry standard) [91]. 

The selection of friction parameters in order to determine the above criteria for evaluating the friction properties was based on the analysis and statistical treatment of the data on the regimes of friction units operation where it is beneficial to use self-lubricated polymeric materials. The characteristic operation regimes were found to be in the range of sliding velocities from 0.01 up to 5.0 m/s and loads from... 

Experimental design and methods Schedule of assessments Subjection selection criteria Screening procedures for entry Study parameters Trial medication Premature withdrawal Subject replacement policy Criteria for excluding data Statistical analysis plans Signatures... 

Defalguerolles, A. and Jmel, S. (1993) Variable selection criteria - based on specific Gaussian graphical models in principal components analysis. Canadian Journal of Statistics - Revue Canadienne de Statistique, 21, 239-56. 

Meta-Analysis. Meta-analysis is a widely used method that employs statistical analysis to synthesize the findings of a number of independent studies. Meta-analyses assume that random variation accounts for differences in results from one study to another. These analyses require both statistical expertise and extensive clinical knowledge of the topic. Weaknesses of meta-analyses are implicit in the process. The included studies may have measured slightly different outcomes, they may have used different research designs, and there may have been selection bias in the criteria used to include a study. 

Another statistic often calculated is an overall index of acceptability of the form (T- Q/(r + C), where T and C are defined as above. This statistic has an expected value of 0 when there is no difference in preference of the test and control treatments and a range from - 1 (when the test treatment is never chosen) to + 1 where the test treatment is completely preferred over the control. While this statistic allows the expression of the relative attraction and deterrence of a range of compounds, its statistical properties are not well understood. The statistic is not normally distributed, and the occurrence of negative values precludes the use of common transformations (e.g., log, arcsine, square root) to remove some deviations from normality. Thus, if this statistic is used, it should be analyzed only by nonparametric, distribution-free statistical tests based on ranks. Further discussion of procedures and criteria for selecting statistical tests can be found in many standard texts and manuals for a number of statistic analysis packages for use with computers (e.g.. Steel Torrie 1980 SAS Institute 1989 Sokal Rohlf 1995). 

---