Unbiased sample estimate

Bias The systematic or persistent distortion of an estimate from the true value. From sampling theory, bias is a characteristic of the sample estimator of the sufficient statistics for the distribution of interest. Therefore, bias is not a function of the data, but of the method for estimating the population statistics. For example, the method for calculating the sample mean of a normal distribution is an unbiased estimator of the true but unknown population mean. Statistical bias is not a Bayesian concept, because Bayes theorem does not relay on the long-term frequency expections of sample estimators. 

The result cited is E[bi] = Pi + P1.2P2 where P1.2 = (Xi Xi) 1X X2, so the coefficient estimator is biased. If the conditional mean function [X2 X1] is a linear function of Xb then the sample estimator P12 actually is an unbiased estimator of the slopes of that function. (That result is Theorem B.3, equation (B-68), in another form). Now, write the model in the form... 

It can be shown that nonlinear LS is at least asymptotically an unbiased MVB estimator in the limit of large sample sizes if the samples are from a population with a normal distribution [60,61]. As pointed out above, the... 

Random sample. One of a sequence of samples (or subsaniples). taken on a random basis to give unbiased statistical estimates. 

So basic is the notion of a statistical estimate of a physical parameter that statisticians use Greek letters for the parameters and Latin letters for the estimates. For many purposes, one uses the variance, which for the sample is s and for the entire populations is cr. The variance s of a finite sample is an unbiased estimate of cr, whereas the standard deviation 5- is not an unbiased estimate of cr. 

Earlier we introduced the confidence interval as a way to report the most probable value for a population s mean, p, when the population s standard deviation, O, is known. Since is an unbiased estimator of O, it should be possible to construct confidence intervals for samples by replacing O in equations 4.10 and 4.11 with s. Two complications arise, however. The first is that we cannot define for a single member of a population. Consequently, equation 4.10 cannot be extended to situations in which is used as an estimator of O. In other words, when O is unknown, we cannot construct a confidence interval for p, by sampling only a single member of the population. 

The second complication is that the values of z shown in Table 4.11 are derived for a normal distribution curve that is a function of O, not s. Although is an unbiased estimator of O, the value of for any randomly selected sample may differ significantly from O. To account for the uncertainty in estimating O, the term z in equation 4.11 is replaced with the variable f, where f is defined such that f > z at all confidence levels. Thus, equation 4.11 becomes... 

If the sample is unbiased, estimate the source mean, so that... 

Figure 65-1 shows a schematic representation of the F-test for linearity. Note that there are some similarities to the Durbin-Watson test. The key difference between this test and the Durbin-Watson test is that in order to use the F-test as a test for (non) linearity, you must have measured many repeat samples at each value of the analyte. The variabilities of the readings for each sample are pooled, providing an estimate of the within-sample variance. This is indicated by the label Operative difference for denominator . By Analysis of Variance, we know that the total variation of residuals around the calibration line is the sum of the within-sample variance (52within) plus the variance of the means around the calibration line. Now, if the residuals are truly random, unbiased, and in particular the model is linear, then we know that the means for each sample will cluster... 

Let 1, x2,..., xn be a random sample of N observations from an unknown distribution with mean fi and variance o2. It can be demonstrated that the sample variance V, given by equation A.8, is an unbiased estimator of the population variance a2. 

This shows that V is an unbiased estimator of a2, regardless of the nature of the sample population. 

Sample mean x and variance s2 are convergent and unbiased estimators (e.g., Hamilton, 1964), which implies that the so-called empirical variance a2 given by... 

We therefore make the assumption that the sample data gathered in vector y are only our best estimates of the real (population) values which justifies the bar on the symbol as representing measured values. This notation contradicts the standard usage, but is consistent with the basic definitions of Chapter 4. Indeed, for an unbiased estimate, we can still write that... 

Note that a scalar behaves as a symmetric matrix.) Because of finite sampling, and P cannot be evaluated exactly. Instead, we will search for unbiased estimates a and P of a and P together with unbiased estimates y( and xtj of yt and xu that satisfy the linear model given by equation (5.4.37) and minimize the maximum-likelihood expression in xt and y,. Introducing m Lagrange multipliers A , one for each linear... 

Bias corrections are sometimes applied to MLEs (which often have some bias) or other estimates (as explained in the following section, [mean] bias occurs when the mean of the sampling distribution does not equal the parameter to be estimated). A simple bootstrap approach can be used to correct the bias of any estimate (Efron and Tibshirani 1993). A particularly important situation where it is not conventional to use the true MLE is in estimating the variance of a normal distribution. The conventional formula for the sample variance can be written as = SSR/(n - 1) where SSR denotes the sum of squared residuals (observed values, minus mean value) is an unbiased estimator of the variance, whether the data are from a normal distribution... 

In general, bias refers to a tendency for parameter estimates to deviate systematically from the true parameter value, based on some measure of the central tendency of the sampling distribution. In other words, bias is imperfect accuracy. In statistics, what is most often meant is mean-unbiasedness. In this sense, an estimator is unbiased (UB) if the average value of estimates (averaging over the sampling distribution) is equal to the true value of the parameter. For example, the mean value of the sample mean (over the sampling distribution of the sample mean) equals the mean for the population. This chapter adheres to the statistical convention of using the term bias (without qualification) to mean mean-unbiasedness. 

The evaluation of hazards posed to human health by toxic airborne chemicals is one of the common tasks employed in industrial hygiene. This process requires the collection of air samples to estimate air concentrations of specific substances inhaled by workers which can then be compared with standards and guides of acceptable exposure. Thus air sampling directly influences the formulation of important decisions. If air samples underestimate exposures, the consequence may be death or occupational disease. Conversely, overestimating exposures may result in the institution of unnecessary controls. Since either form of error is undesirable, it is fundamentally important that air sampling accurately define the extent of hazard. This requires that air samples be collected according to scientific, unbiased schemes for estimating exposures to toxic airborne chemicals. 

The parameters A,k and b must be estimated from sr The general problem of parameter estimation is to estimate a parameter, 0, given a number of samples, x,-, drawn from a population that has a probability distribution P(x, 0). It can be shown that there is a minimum variance bound (MVB), known as the Cramer-Rao inequality, that limits the accuracy of any method of estimating 0 [55]. There are a number of methods that approach the MVB and give unbiased estimates of 0 for large sample sizes [55]. Among the more popular of these methods are maximum likelihood estimators (MLE) and least-squares estimation (LS). The MLE... 

The definition of sample variance with an (n-1) in the denominator leads to an unbiased estimate of the population variance, as shown above. Sometimes the sample variance is defined as the biased variance ... 

An estimate is unbiased if on the average it predicts the correct value. Mathematically, an estimate is unbiased if its expected value is equal to the population parameter that it is estimating. For example, the sample mean X is an unbiased estimate of the population mean p because ... 

We have also seen that X is an unbiased, efficient, consistent estimate of p, if the sample is from an underlying normal population. If the underlying population deviates substantially from normality, the mean may not be the efficient estimate and some other measure of location such as the median may be preferable. We have previously illustrated a simple test on the mean with an underlying normal population of known variance. We shall review this case briefly, applying it to tests between two means, and then proceed to tests where the population variance is unknown. 

We note from Table 1.19 that the sums of squares between rows and between columns do not add up to the defined total sum of squares. The difference is called the sum of squares for error, since it arises from the experimental error present in each observation. Statistical theory shows that this error term is an unbiased estimate of the population variance, regardless of whether the hypotheses are true or not. Therefore, we construct an F-ratio using the between-rows mean square divided by the mean square for error. Similarly, to test the column effects, the F-ratio is the be-tween-columns mean square divided by the mean square for error. We will reject the hypothesis of no difference in means when these F-ratios become too much greater than 1. The ratios would be 1 if all the means were identical and the assumptions of normality and random sampling hold. Now let us try the following example that illustrates two-way analysis of variance. 

A normally distributed parent population X is characterized by its expected mean value /< and standard deviation a. If the samples taken from this population are representative, the sample average x is an unbiased estimate of /c... 

Dividing the sample sum of squares by the degree of freedom (n — 1) yields an unbiased estimate. If all observations are equal, then there is no variability and s2 = 0. The sample variance becomes increasingly large as the amount of variability or dispersion increases. 

Probability distribution models can be used to represent frequency distributions of variability or uncertainty distributions. When the data set represents variability for a model parameter, there can be uncertainty in any non-parametric statistic associated with the empirical data. For situations in which the data are a random, representative sample from an unbiased measurement or estimation technique, the uncertainty in a statistic could arise because of random sampling error (and thus be dependent on factors such as the sample size and range of variability within the data) and random measurement or estimation errors. The observed data can be corrected to remove the effect of known random measurement error to produce an error-free data set (Zheng Frey, 2005). 

There are often data sets used to estimate distributions of model inputs for which a portion of data are missing because attempts at measurement were below the detection limit of the measurement instrument. These data sets are said to be censored. Commonly used methods for dealing with such data sets are statistically biased. An example includes replacing non-detected values with one half of the detection limit. Such methods cause biased estimates of the mean and do not provide insight regarding the population distribution from which the measured data are a sample. Statistical methods can be used to make inferences regarding both the observed and unobserved (censored) portions of an empirical data set. For example, maximum likelihood estimation can be used to fit parametric distributions to censored data sets, including the portion of the distribution that is below one or more detection limits. Asymptotically unbiased estimates of statistics, such as the mean, can be estimated based upon the fitted distribution. Bootstrap simulation can be used to estimate uncertainty in the statistics of the fitted distribution (e.g. Zhao Frey, 2004). Imputation methods, such as... 

---