Minimum-variance unbiased estimator

Unbiased and Minimum-Variance Unbiased Estimation, Particularly for Variances... 

The linear minimum variance unbiased estimate C of C given data A under these conditions is according to Gauss-Markoff theorem equal to... 

If it is assumed that the measurement errors are normally distributed, the resolution of problem (5.3) gives maximum likelihood estimates of process variables, so they are minimum variance and unbiased estimators. 

If the errors are normally distributed, the OLS estimates are the maximum likelihood estimates of 9 and the estimates are unbiased and efficient (minimum variance estimates) in the statistical sense. However, if there are outliers in the data, the underlying distribution is not normal and the OLS will be biased. To solve this problem, a more robust estimation methods is needed. 

Prove that the least squares intercept estimator in the classical regression model is the minimum variance linear unbiased estimator. 

The parameters A,k and b must be estimated from sr The general problem of parameter estimation is to estimate a parameter, 0, given a number of samples, x,-, drawn from a population that has a probability distribution P(x, 0). It can be shown that there is a minimum variance bound (MVB), known as the Cramer-Rao inequality, that limits the accuracy of any method of estimating 0 [55]. There are a number of methods that approach the MVB and give unbiased estimates of 0 for large sample sizes [55]. Among the more popular of these methods are maximum likelihood estimators (MLE) and least-squares estimation (LS). The MLE... 

The general least-squares treatment requires that the generalized sum of squares of the residuals, the variance a2, be minimized. This is, by the geometry of error space, tantamount to the requirement that the residual vector be orthogonal with respect to fit space, and this is guaranteed when the scalar products of all fit vectors (the rows of XT) with the residual vector vanish, XTM 1 = 0, where M 1 is the metric of error space. The successful least-squares treatment [34] yields the following minimum-variance linear unbiased estimators (A) for the variables, their covariance matrix, the variance of the fit, the residuals, and their covariance matrix ... 

Also under OLS assumptions, the regression parameter estimates have a number of optimal properties. First, 0 is an unbiased estimator for 0. Second, the standard error of the estimates are at a minimum, i.e., the standard error of the estimates will be larger than the OLS estimates given any other assumptions. Third, assuming the errors to be normally distributed, the OLS estimates are also the maximum likelihood (ML) estimates for 0 (see below). It is often stated that the OLS parameter estimates are BLUE (Best Linear Unbiased Predictors) in the sense that best means minimum variance. Fourth, OLS estimates are consistent, which in simple terms means that as the sample size increases the standard error of the estimate decreases and the bias of the parameter estimates themselves decreases. 

With linear models, exact inferential procedures are available for any sample size. The reason is that as a result of the linearity of the model parameters, the parameter estimates are unbiased with minimum variance when the assumption of independent, normally distributed residuals with constant variance holds. The same is not true with nonlinear models because even if the residuals assumption is true, the parameter estimates do not necessarily have minimum variance or are unbiased. Thus, inferences about the model parameter estimates are usually based on large sample sizes because the properties of these estimators are asymptotic, i.e., are true as n —> oo. Thus, when n is large and the residuals assumption is true, only then will nonlinear regression parameter estimates have estimates that are normally distributed and almost unbiased with minimum variance. As n increases, the degree of unbiasedness and estimation variability will increase. 

The expected value of bQ = E[bo =j3o- The expected value of bi =E[bi] =j3i-The least-squares estimators of bo and b are unbiased estimators and have the minimum variance of all other possible linear combinations. 

Once a transformation is determined for the regression, substitute y for y and plot the residuals. The process is an iterative one. It is particularly important to correct a nonconstant when providing confidence intervals for prediction. The least squares estimator will still be unbiased, but no Imiger for a minimum variance probability. 

A weU-known procedure for finding estimators of unknown parameters is the method of maximum likelihood (Devore 1987 Dougherty 1990 Hogg and Craig 1978). Maximum-likelihood estimators are consistent and have minimum variance but are not always unbiased. A summary of the procedure follows. 

The least squares estimation in model (14), (15) leads to the unbiased, minimum variance estimators... 

An unbiased, minimum variance estimator is also a minimum mean square error estimator. 

Since Theorem 6.3 states that the prediction error method produces unbiased estimates, as m —> oo, the estimated parameter values will approach the true parameter values. Thus, it can be concluded that the prediction error method asymptotically approaches a minimum variance estimator. 

The gain matrices Mpt] and Lptj are determined such that both the input estimates p i and the state estimates X i i ij are minimum variance and unbiased (i.e M i = argmuiM tr Ppjj jj, ... 

Gillijns S, De Moor B (2007) Unbiased minimum-variance input and state estimation for linear discrete-time systeans with direct feedthrough. Aubnnatica 43(5) 934—937... 

It can be shown that Pp is an unbiased and consistent estimator of the probability of failure with minimum variance for a specified Ni. Furthermore, the sampling variance here is given... 

Figure 1 shows a block diagram for the perturbed state of a robot, e, subject to both the process noise w and measurement noise y. The actually measured perturbed state is denoted as z. The Kalman filter is the best linear estimator in the sense that it produces unbiased, minimum variance estimates (Kalman and Bucy, 1961 Brown, 1983). Let (t) be the estimated perturbed state and 6eg(t) be the residual which is the difference between the true measured perturbed state, z(t), and the estimated perturbed state based on 6a (t), here denoted as (t). It has already been shown (Lewis, 1986) that Cx satisfies a differential equation which can be schematically represented by the block diagram shown in Fig. 2 where K(t) is a Kalman filter gain. K(t) is to be calculated according to the equation... 

---