Residual Variance Model Parameter Estimation Using Weighted Least-Squares

RESIDUAL VARIANCE MODEL PARAMETER ESTIMATION USING WEIGHTED LEAST-SQUARES 

Carroll and Ruppert (1988) and Davidian and Gil-tinan (1995) present comprehensive overviews of parameter estimation in the face of heteroscedasticity. In general, three methods are used to provide precise, unbiased parameter estimates weighted least-squares (WLS), maximum likelihood, and data and/or model transformations. Johnston (1972) has shown that as the departure from constant variance increases, the benefit from using methods that deal with heteroscedasticity increases. The difficulty in using WLS or variations of WLS is that additional burdens on the model are made in that the method makes the additional assumption that the variance of the observations is either known or can be estimated. In WLS, the goal is not to minimize the OLS objective function, i.e., the residual sum of squares, 

When the variance of an observation is large, less weight is given to that observation. Conversely, when an observation is precisely measured with small variance, more weight is given to that observation, a beneficial property. Obviously, ordinary least-squares is a special case of weighted least-squares in that all the observations have weights equal to one. 

The choice of weights in WLS is based on either the observed data or on a residual variance model. If the variances are known and they are not a function of the mean, then the model can be redefined as 

If replicate measurements are taken at each level of the predictors, then one simple approach to estimation of the weights is to calculate the sample variance at each level i of the predictors (of) and use these to form the weights as 

---