Unbiased estimate of the population varianc

Let 1, x2,..., xn be a random sample of N observations from an unknown distribution with mean fi and variance o2. It can be demonstrated that the sample variance V, given by equation A.8, is an unbiased estimator of the population variance a2. 

The definition of sample variance with an (n-1) in the denominator leads to an unbiased estimate of the population variance, as shown above. Sometimes the sample variance is defined as the biased variance ... 

If we desire to study the effects of two independent variables (factors) on one dependent factor, we will have to use a two-way analysis of variance. For this case the columns represent various values or levels of one independent factor and the rows represent levels or values of the other independent factor. Each entry in the matrix of data points then represents one of the possible combinations of the two independent factors and how it affects the dependent factor. Here, we will consider the case of only one observation per data point. We now have two hypotheses to test. First, we wish to determine whether variation in the column variable affects the column means. Secondly, we want to know whether variation in the row variable has an effect on the row means. To test the first hypothesis, we calculate a between columns sum of squares and to test the second hypothesis, we calculate a between rows sum of squares. The between-rows mean square is an estimate of the population variance, providing that the row means are equal. If they are not equal, then the expected value of the between-rows mean square is higher than the population variance. Therefore, if we compare the between-rows mean square with another unbiased estimate of the population variance, we can construct an F test to determine whether the row variable has an effect. Definitional and calculational formulas for these quantities are given in Table 1.19. 

We note from Table 1.19 that the sums of squares between rows and between columns do not add up to the defined total sum of squares. The difference is called the sum of squares for error, since it arises from the experimental error present in each observation. Statistical theory shows that this error term is an unbiased estimate of the population variance, regardless of whether the hypotheses are true or not. Therefore, we construct an F-ratio using the between-rows mean square divided by the mean square for error. Similarly, to test the column effects, the F-ratio is the be-tween-columns mean square divided by the mean square for error. We will reject the hypothesis of no difference in means when these F-ratios become too much greater than 1. The ratios would be 1 if all the means were identical and the assumptions of normality and random sampling hold. Now let us try the following example that illustrates two-way analysis of variance. 

The sample should provide an unbiased estimate of the population variance, so that tests of significance may be applied. This objective is achieved only if every possible unit of a preselected size has an equal chance of being drawn. 

A calculation of this sort ensures that the measure of dispersion is positive (squaring the deviations ensures that) and dividing by (n - 1) results in a quantity that represents an average of sorts. The sample variance is the "typical" or "average" squared deviation of observations from the sample mean. The use of the (n - 1) in the denominator may seem confusing, but the reason why this is done is that calculating the sample variance in this manner yields an unbiased estimator of the population variance, which is represented by the symbol o-. (The exact mathematical... 

Note The standard deviation s will be obtained by taking the square root of I2, although it happens that whereas I2 is an unbiased estimate of the population variance, is not an unbiased estimate of the population standard deviation.] (The term unbiased indicates that n — 1 is used in the denominator instead of n, in defining variance. In order to avoid repetition from here on, the word unbiased will not be used although all... 

Suppose that in a set of n values of b, each has a known population variance, erf. The sample variance, (defined by Eq. (18.2)) is actually an unbiased estimate of the population variance, a , which is defined as... 

The pooled estimator of the population variance, Sp, is an unbiased estimator for o2 regardless of whether the population means ilt [t2,..., [t[ are equal or not, because it takes into account deviations from each group mean X.j, j=l, 2,..., J. Unbiasedness follows from Eq. (1.114) since ... 

If the group population means are all equal, then is an unbiased estimate of the variance of the population mean o. To obtain an estimator of the population variance o2 recall that ... 

Note that the term (n - 1) is used in the denominator of this equation to ensure that 5 is an unbiased estimate of the population standard deviation, a. (There is a general convention in statistics that English letters are used to describe the properties of samples, Greek letters to describe populations). The term (n - 1) is the number of degrees of freedom of the estimate, s. This is because if x is known, it is only necessary to know the values of (n — 1) of the individual measurements, as by definition S(.x,- — x) = 0. The square of the standard deviation, is known as the variance, and is a very important statistic when two or more sources of error are being considered, because of its additivity properties. 

This simply means that the numbers of samples within strata should be proportional to the sizes of the strata. This procedure is commonly called representative sampling, it gives an unbiased estimate of the population mean, but leads to a larger variance of the estimate than the procedure represented by Equations (27-7) and (27-8) unless the variance is uniform in all the strata. 

Global Two-Stage Method. An extensive description of the method is provided by Steimer et al. The global two-stage (GTS) approach has been shown, through simulation, to provide unbiased estimates of the population mean parameters and their variance-covariance, whereas the estimates of the variances were upwardly biased if the STS approach was used. These simulations were done under the ideal situation that the residual error was normally distributed with a known variance. However, it is a well-known fact that the asymptotic covariance matrix used in the calculations is approximate and under less ideal conditions, the approximation can be poor. ... 

To obtain a variance that is an unbiased estimate of the population varianee so that valid confidence limits can be found for the mean, and various hypothesis tests can be applied. This goal can be reached only if every possible sample is equally likely to be drawn. 

We use the symbol si for a sample variance and the symbol al for a population variance. Statisticians have shown that if we use the A - 1 denominator in Eq. (15.59) then the sample variance si is an unbiased estimate of the population... 

When the subpopulations or strata are unequal in size and in variance, it can be shown that, if an estimate of the population mean is to be unbiased and the variance of the estimate is to be minimal, the number of samples taken from each stratum should be proportional to the size of the stratum and also to its standard deviation, or... 

Mixed-effects models, which will be described in later chapters, do not suffer from this flaw and tend to produce both unbiased mean and variance estimates. As an example, Sheiner and Beal (1980) used Monte Carlo simulation to study the accuracy of the two-stage approach and mixed effects model approach in fitting an Emax model with parameters Vmax, Km - Data from 49 individuals were simulated. The relative deviation from the mean estimated value to the true simulated value for Vmax and Km was 3.7% and —4.9%, respectively, for the two-stage method and —0.9 and 8.3%, respectively, for the mixed effects model approach. Hence, both methods were relatively unbiased in their estimation of the population means. However, the relative deviation from the mean estimated variance to the true simulated variance for Vmax and Km was 70 and 82%, respectively, for the two-stage method and —2.6 and 4.1%, respectively, for the mixed effects model approach. Hence, the variance components were significantly overestimated with the two-stage approach. Further, the precision of the estimates across simulations tended to be more variable with the two-stage approach than with the mixed effects... 

Notice that the true average square deviation from the mean would be obtained by dividing by n rather than n — 1 as shown in Eq. (5.2). Statistical theory [2] will show that if division is by n, the value of Sj so obtained will notht an unbiased estimate of the true population variance a. TTie distinction is particularly important when only a small number (around 10) of measurements are being made. 

The sample variance, s, is an unbiased estimate (see Sect. 20.2.1) of the population variance, with v degrees of freedom. The statistic vs has the chi-square distribution with V degrees of freedom. The lOO(l-a) % two-sided confidence interval is ... 

So basic is the notion of a statistical estimate of a physical parameter that statisticians use Greek letters for the parameters and Latin letters for the estimates. For many purposes, one uses the variance, which for the sample is s and for the entire populations is cr. The variance s of a finite sample is an unbiased estimate of cr, whereas the standard deviation 5- is not an unbiased estimate of cr. 

The parameters A,k and b must be estimated from sr The general problem of parameter estimation is to estimate a parameter, 0, given a number of samples, x,-, drawn from a population that has a probability distribution P(x, 0). It can be shown that there is a minimum variance bound (MVB), known as the Cramer-Rao inequality, that limits the accuracy of any method of estimating 0 [55]. There are a number of methods that approach the MVB and give unbiased estimates of 0 for large sample sizes [55]. Among the more popular of these methods are maximum likelihood estimators (MLE) and least-squares estimation (LS). The MLE... 

We have also seen that X is an unbiased, efficient, consistent estimate of p, if the sample is from an underlying normal population. If the underlying population deviates substantially from normality, the mean may not be the efficient estimate and some other measure of location such as the median may be preferable. We have previously illustrated a simple test on the mean with an underlying normal population of known variance. We shall review this case briefly, applying it to tests between two means, and then proceed to tests where the population variance is unknown. 

Reproducibility, or precision, of a method relates to how individual estimates fluctuate around the average value. The magnitude of the fluctuation in the population is expressed by the parameter variance Variance is the average of the squared deviations about p for all values x, in the population E(x, f/N. An unbiased estimate of is obtained from the deviation of each value (x,) around the mean (x ) for a sample taken from the population = S(x, — x) /... 

This is known as the biased variance. It is a good measure of the dispersion of a sample of a flow variable, but according to the statistical definition not the best measure of the dispersion of the whole population of possible observations. A better estimate of the variance (an unbiased variance) of the population, given a sample of data, is ... 

---