Sampling error sample size

In the next several sections, the theoretical distributions and tests of significance will be examined beginning with Student s distribution or t test. If the data contained only random (or chance) errors, the cumulative estimates x and 5- would gradually approach the limits p and cr. The distribution of results would be normally distributed with mean p and standard deviation cr. Were the true mean of the infinite population known, it would also have some symmetrical type of distribution centered around p. However, it would be expected that the dispersion or spread of this dispersion about the mean would depend on the sample size. 

Vitha, M. F. Carr, P. W. A Laboratory Exercise in Statistical Analysis of Data, /. Chem. Educ. 1997, 74, 998-1000. Students determine the average weight of vitamin E pills using several different methods (one at a time, in sets of ten pills, and in sets of 100 pills). The data collected by the class are pooled together, plotted as histograms, and compared with results predicted by a normal distribution. The histograms and standard deviations for the pooled data also show the effect of sample size on the standard error of the mean. 

Instead of calculations, practical work can be done with scale models (33). In any case, calculations should be checked wherever possible by experimental methods. Using a Monte Carlo method, for example, on a shape that was not measured experimentaUy, the sample size in the computation was aUowed to degrade in such a way that the results of the computation were inaccurate (see Fig. 8) (30,31). Reversing the computation or augmenting the sample size as the calculation proceeds can reveal or eliminate this source of error. 

The precondition for the use of the normal distribution in estimating the random error is that adequate reliable estimates are available for the parame-rcrs ju. and cr. In case of a repeated measurement, the estimates are calculated using Eqs. (12.1) and (12,3). When the sample size iiicrease.s, the estimates m and s approach the parameters /c and cr. A rule of rhumb is that when s 30. the normal distribution can be osecl,... 

A measure of variability of the estimate can be gained from the standard error but it can be seen from Equations 11.4 and 11.5 that the magnitude of the standard error is inversely proportional to n (i.e., the larger the sample size the smaller will be the standard error). Therefore, without... 

A problem long appreciated in economic evaluations, but whose seriousness has perhaps been underestimated (Sturm et al, 1999), is that a sample size sufficient to power a clinical evaluation may be too small for an economic evaluation. This is mainly because the economic criterion variable (cost or cost-effectiveness) shows a tendency to be highly skewed. (One common source of such a skew is that a small proportion of people in a sample make high use of costly in-patient services.) This often means that a trade-off has to be made between a sample large enough for a fully powered economic evaluation, and an affordable research study. Questions also need to be asked about what constitutes a meaningful cost or cost-effectiveness difference, and whether the precision (type I error) of a cost test could be lower than with an effectiveness test (O Brien et al, 1994). 

The sample size (a function of its cr5retedlinity) is usually determined by trial and error. About 500 milligrams is usually sufficient. 

For all degrees of peak overlap it is well established that peak height is a more accurate measure of sample size than peak area for symmetrical peaks (277,279). For either method the error increases for disproportionate peak sizes. For tailed peaks errors in either peak height or area can assume large proportions using perpendicular drop method (279,281). In general, the most... 

Wet decomposition in open vessels allows large sample sizes and achieves low detection limits, but may give rise to systematic errors due to ... 

The experimental design used was nested plots. Main plots had a total area of 100 m2 and were rectangular plots. The criterion used to determine sample size for each stratum was an estimation of AGB of trees with a diameter at breast height (dbh) >10 cm during pre-sampling (90% probability and 20% mean standard error). 

The principle of sequential analysis consists of the fact that, when comparing two different populations A and B with pre-set probabilities of risks of error, a and /3, just as many items (individual samples) are examined as necessary for decision making. Thus the sample size n itself becomes a random variable. 

Wood, R. H. Muhlbauer, W. C. F. Thompson, P. T., Systematic errors in free energy perturbation calculations due to a finite sample of configuration space. Sample-size hysteresis, J. Phys. Chem. 1991, 95, 6670-6675... 

In the equations above, the mean square error, the sample variance, and the finite sampling bias are all explicitly written as functions of the sample size N. Both the variance and bias diminish as /V — oc (infinite sampling). However, the variance... 

The sample size in a real simulation is always finite, and usually relatively small. Thus, understanding the error behavior in the finite-size sampling region is critical for free energy calculations based on molecular simulation. Despite the importance of finite sampling bias, it has received little attention from the community of molecular simulators. Consequently, we would like to emphasize the importance of finite sampling bias (accuracy) in this chapter. 

What level of inaccuracy can be expected for a simulation with a certain sample size N1 This question can be transformed to another one what is the effective limit-perturbation Xf or xg in the inaccuracy model [(6.22) or (6.23)] To assess the error in a free energy calculation using the model, one may histogram / and g using the perturbations collected in the simulations, and plot x in the tail of the distribution. However, if Xf is taken too small the accuracy is overestimated, and the assessed reliability of the free energy is therefore not ideal. In the following, we discuss the most-likely analysis, which provides a more systematic way to estimate the accuracy of free energy calculations. 

These bounds originate from the systematic errors (biases) due to the finite sampling in free energy simulations and they differ from other inequalities such as those based on mathematical statements or the second law of thermodynamics. The bounds become tighter with more sampling. It can be shown that, statistically, in a forward calculation AA(M) < AA(N) for sample sizes M and N and M > N. In a reverse calculation, AA(M) > AA(N). In addition, one can show that the inequality (6.27) presents a tighter bound than that of the second law of thermodynamics... 

In practice it is helpful to know the order of magnitude of the sample size N needed to reach a reasonably accurate free energy. The inaccuracy model described above presents an effective way to relate the sample size N and the finite sampling error through perturbation distribution functions. Alternatively, one can develop a heuristic that does not involve distribution functions and is determined by exploring the common behavior of free energy calculations for different systems [25]. Although only FEP calculations are considered in this section, the analysis extends to NEW calculations. 

Fig. 6.7. The relative error versus the number of samples in several model systems. Plotting the error versus sample size directly creates a distribution of convergence rates. See [25] for more details on the systems...

Now consider the finite sampling systematic error. As discussed in Sect. 6.4.1, the fractional bias error in free energy is related to both the sample size and entropy difference 5e N exp(-AS/kB). With intermediates defined so that the entropy difference for each substage is the same (i.e., AS/n), the sampling length Ni required to reach a prescribed level of accuracy is the same for all stages, and satisfies... 

Fig. 6.10. Comparison of overlap sampling and FEP calculation results for the free energy change along the mutation of an adenosine in aqueous solution (between A = 0.05 and 0.45) in a molecular dynamics simulation. The results represent the average behavior of 14 independent runs. (MD time step.) The sampling interval is 0.75 ps. The upper half of the plot presents the standard deviation of the mean (with gives statistical error) for AA as a function of sample size N the lower half of the plot gives the estimate of A A - for comparison of the accuracy, the correct value of AA is indicated by the bold horizontal line...

---