Heteroscedastic error

In case that the measuring errors sy vary and therefore heteroscedasticity must be assumed, the original LS criterion (Eq. 6.14) must be applied and the model of weighted least squares (WLS) results from this criterion as will be shown in Sect. 6.2.3. 

The methods used were those of Mitchell ( 1 ), Kurtz, Rosenberger, and Tamayo ( 2 ), and Wegscheider T ) Mitchell accounted for heteroscedastic error variance by using weighted least squares regression. Mitchell fitted a curve either to all or part of the calibration range, using either a linear or a quadratic model. Kurtz, et al., achieved constant variance by a... 

A graphical display of the residuals tells us a lot about our data. They should be normally distributed (top left). If the variances increase with the concentration, we have inhomogeneous variances, called heteroscedasticity (bottom left). The consequences are discussed in the next slide. If we have a linear trend in the residuals, we probably used the wrong approach or we have a calculation error in our procedure (top right). Non-linearity of data deliver the situation described on bottom right, if we nevertheless use the linear function. 

These and most other equations developed by statisticians assume that the experimental error is the same over the entire response surface there is no satisfactory agreement for how to incorporate heteroscedastic errors. Note that there are several different equations in the literature according to the specific aims of the confidence interval calculations, but for brevity we introduce only two which can be generally applied to most situations. 

Finally it is often useful to be able estimate the experimental error (as discussed in Section 2.2.2), and one method is to perform extra replicates (typically five) in the centre. Obviously other approaches to replication are possible, but it is usual to replicate in the centre and assume that the error is the same throughout the response surface. If there are any overriding reasons to assume that heteroscedasticity of errors has an important role, replication could be performed at the star or factorial points. However, much of experimental design is based on classical statistics where there is no real detailed information about error distributions over an experimental domain, or at least obtaining such information would be unnecessarily laborious. 

Impedance measurements are, in general, heteroscedastic, which means that the variance of the stochastic errors is a strong function of frequency. It is important, therefore, to use a weighting strategy that accoimts for the frequency dependence of the stochastic errors. 

The response error (A/ ) is not constant (heteroscedastic) so that the highest precision does not necessarily coincide with the highest sensitivity (Section 15.4). 

The assumption of homoscedasticity means that the residual variability should be constant over all available data dimensions (predictions, covariates, time, etc). If we observe heteroscedasticity, then we need to change the residual error model to account for this. In practice, this means that we should weight the data differently by using a different model for the residual variability. 

A method close to the IT2S procedure is the expectation-maximization-like (EM) method presented by Mentre and Geomeni (36), which can be viewed as an extension of the IT2S procedure when both random and fixed effects are included in the model and for heteroscedastic errors known to a proportionality coefficient. This algorithm is implemented with the software P-PHARM (37). 

Faber NM, Lorber A, Kowalski BR, Generalized rank annihilation method Standard errors in the estimated eigenvalues if the instrumental errors are heteroscedastic and correlated, Journal of Chemometrics, 1997,11, 95-109. 

The data are known as homoscedastic, which means that the errors in y are independent of the concentration. Data for which the uncertainty, for example, grows with the concentration are heteroscedastic data. 

Notice that nothing beyond the first two moments of Y is being assumed, i.e., only the mean and variance of the data are being defined and no distributional assumptions, such as normality, are being made. In residual variance model estimation, the goal is to understand the variance structure as a function of a set of predictors, which may not necessarily be the same as the set of predictors in the structural model (Davidian and Car-roll, 1987). Common, heteroscedastic error models are shown in Table 4.1. Under all these models, generic s is assumed to be independent, having zero mean and constant variance. 

One method for dealing with heteroscedastic data is to ignore the variability in Y and use unweighted OLS estimates of 0. Consider the data shown in Fig. 4.2 having a constant variance plus proportional error model. The true values were volume of distribution = 10 L, clearance = 1.5 L/h, and absorption rate constant = 0.7 per/h. The OLS estimates from fitting a 1-compartment model to the data were as follows volume of distribution = 10.3 0.15L, clearance = 1.49 0.01 L/h, and absorption rate constant =0.75 0.03 per h. The parameter estimates themselves were quite well estimated, despite the fact that the assumption of constant variance was violated. Figure 4.3 presents the residual plots discussed in the previous section. The top plot, raw residuals versus predicted values, shows that as the predicted values increase so do the variance of the residuals. This is confirmed by the bottom two plots of Fig. 4.3 which indicate that both the range of the absolute value of the residuals and squared residuals increase as the predicted values increase. 

Long and Ervin (2000) used Monte Carlo simulation to compare the four estimators under a homoscedastic and heteroscedastic linear model. The usually reported standard error estimator [Eq. (4.10)] was not studied. All heteroscedastic estimators of performed well even when heteroscedasticity was not present. When heteroscedasticity was present, Eq. (4.11) resulted in incorrect inferences when the sample size was less than 250 and was also more likely to result in a Type I error than the other estimators. When more than 250 observations were present, all estimators performed equally. Long and Ervin suggest that when the sample size is less than 250 that Eq. (4.16) be used to estimate . Unfortunately no pharmacokinetic and most statistical software packages use these heteroscedastic-consistent standard error estimators. 

In summary, the Type I error rate from using the LRT to test for the inclusion of a covariate in a model was inflated when the data were heteroscedastic and an inappropriate estimation method was used. Type I error rates with FOCE-I were in general near nominal values under most conditions studied and suggest that in most cases FOCE-I should be the estimation method of choice. In contrast, Type I error rates with FO-approximation and FOCE were very dependent on and sensitive to many factors, including number of samples per subject, number of subjects, and how the residual error was defined. The combination of high residual variability with sparse sampling was a particularly disastrous combination using... 

Long, J.S. and Ervin, L.H. Using heteroscedasticity consistent standard errors in the linear regression model. American Statistician 2000 54 217-224. 

Parks, R.E. Estimation with heteroscedastic error terms. Econ-ometrica 1966 34 888. 

The residuals may be distributed in various different ways. First of all they may be scattered more or less symmetrically about zero. This dispersion can be described by a standard deviation of random experimental error. If this is (approximately) constant over the experimental region the system is homoscedastic, as has been assumed up to now. However the analysis of residuals may show that the standard deviation varies within the domain, and the system is heteroscedastic. On the other hand it may reveal systematic errors where the residuals are not distributed symmetrically about zero, but show trends which indicate model inadequacy. 

In the following section we describe some of these methods and how they may show the different effects of dispersion and systematic error. Then in the remaining two sections of the chapter we will discuss methods for treating heteroscedastic systems. In the first place, we will show how their non-constant standard deviation may be taken into account in estimating models for the kind of treatment we have already described. Then we will describe the detailed study of dispersion within a domain, often employed to reduce variation of a product or process. 

Assumes a heteroscedastic error structure (variance changes widi the response). The random error is assumed to be some function of the observed data (i.e., if Wi = 1 /, the variance is... 

Assumes a heteroscedastic error structure. The variance is expressed as a model parameter in conjunction with die structural model parameters. ELS is designated as a maximum likelihood (as opposed to least squares) if the random effects are assumed to be normally distributed. 

Compare proportional vs. additive random error models examine plots that assess accounting of heteroscedasticity (see Diagnostics). 

It should be stressed that, depending on the size of the errors we are willing to tolerate in the predictions made from the regression equation, it might be that neither lack of fit nor response heteroscedasticity have any practical importance. In any case, it is well to be prepared to treat... 

If the experimental error is comparably large for all factor combinations, the data are termed homoscedastic. In the case of heteroscedastic data, the errors differ at different factor combinations. 

A second objection to using the line of regression of y on x, as calculated in Sections 5.4 and 5.5, in the comparison of two analytical methods is that it also assumes that the error in the y-values is constant. Such data are said to be homoscedastic. As previously noted, this means that all the points have equal weight when the slope and intercept of the line are calculated. This assumption is obviously likely to be invalid in practice. In many analyses, the data are heteroscedastic, i.e. the standard deviation of the y-values increases with the... 

---