Residual Variance Models

Equation (4.2) is called a residual variance model, but it is not a very general one. In this case, the model states that random, unexplained variability is a constant. Two methods are usually used to estimate 0 least-squares (LS) and maximum likelihood (ML). In the case where e N(0, a2), the LS estimates are equivalent to the ML estimates. This chapter will deal with the case for more general variance models when a constant variance does not apply. Unfortunately, most of the statistical literature deals with estimation and model selection theory for the structural model and there is far less theory regarding choice and model selection for residual variance models. 

Equation (4.4) is a residual variance model like Eq. (4.2) but far more flexible. Most often the residual variance model is simply a function of the structural model f(0 x) in which case Eq. (4.4) is simplified to... 

Notice that nothing beyond the first two moments of Y is being assumed, i.e., only the mean and variance of the data are being defined and no distributional assumptions, such as normality, are being made. In residual variance model estimation, the goal is to understand the variance structure as a function of a set of predictors, which may not necessarily be the same as the set of predictors in the structural model (Davidian and Car-roll, 1987). Common, heteroscedastic error models are shown in Table 4.1. Under all these models, generic s is assumed to be independent, having zero mean and constant variance. 

Usually, variability increases as a systematic function of the mean response f(0 x) in which case a common choice of residual variance model is the power of the mean model... 

Figure 4.1 Variance of Y as a function of the mean response assuming a power of the mean residual variance model.

Davidian and Haaland (1990) call this a components of variance model because is the variance component from the detector when no analyte is present and f(x 0)4>2 is the variance component from the analyte response. Residual variance models can be empirically determined by trial and error, and indeed most are chosen in this manner, but hopefully they will be selected based on the scientist s knowledge of the process. A full description of variance model estimation is made in Davidian and Carroll (1987) and Carroll et al. (1988). 

In contrast, the data in the top plot of Fig. 4.2 using a constant residual variance model led to the following parameter estimates after fitting the same model volume of distribution =10.2 0.10L, clearance = 1.49 0.008 L/h, and absorption rate constant = 0.71 0.02 per h. Note that this model is the data generating model with no regression assumption violations. The residual plots from this analysis are shown in Fig. 4.4. None of the residual plots show any trend or increasing variance with increasing predicted value. Notice that the parameter estimates are less biased and have smaller standard errors than the estimates obtained from the constant variance plus proportional error model. 

This is the effect of violating the assumptions of OLS the parameter estimates have decreased precision when the incorrect residual variance model is used. The precision of the predicted values is also sacrificed. In... 

RESIDUAL VARIANCE MODEL PARAMETER ESTIMATION USING WEIGHTED LEAST-SQUARES... 

The choice of weights in WLS is based on either the observed data or on a residual variance model. If the variances are known and they are not a function of the mean, then the model can be redefined as... 

However, when the number of replicates is small, as is usually the case, the estimated variance can be quite erroneous and unstable. Nonlinear regression estimates using this approach are more variable than their unweighted least-squares counterparts, unless the number of replicates at each level is 10 or more. For this reason, this method cannot be supported and the danger of unstable variance estimates can be avoided if a parametric residual variance model can be found. 

The particular choice of a residual variance model should be based on the nature of the response function. Sometimes 4> is unknown and must be estimated from the data. Once a structural model and residual variance model is chosen, the choice then becomes how to estimate 0, the structural model parameters, and <, the residual variance model parameters. One commonly advocated method is the method of generalized least-squares (GLS). First it will be assumed that < is known and then that assumption will be relaxed. In the simplest case, assume that 0 is known, in which case the weights are given by... 

Another common fitting algorithm found in the pharmacokinetic literature is extended least-squares (ELS) wherein 0, the structural model parameters, and 4>, the residual variance model parameters, are estimated simultaneously (Sheiner and Beal, 1985). The objective function in ELS is the same as the objective function in PL... 

Building on work done earlier, Beal and Sheiner (1988) used Monte Carlo simulation to compare the parameter estimates obtained from ELS, GLS, WLS, OLS, and a few other modifications thereof. The models studied were the 1-compartment model, 1-compartment model with absorption, and Emax model. Each model was evaluated under five different residual variance models. All the methods used to deal with heteroscedas-ticity were superior to OLS estimates. There was little difference between estimates obtained using GLS and... 

ELS and that earlier enthusiasm expressed for ELS must be tempered. Two other notable observations were made. First, if the residual variance model was incorrectly specified with ELS, this adversely affected the ELS parameter estimates. Belanger et al. (1971) indicated that this should not be a problem with GLS. Second, considerable advantage was gained when was treated as an estimable parameter, rather than a fixed value as in IRWLS or WLS, as estimable residual variance models are more robust to residual variance model misspecification than fixed residual variance models. 

As a last comment, caution should be exercised when fitting small sets of data to both structural and residual variance models. It is commonplace in the literature to fit individual data and then apply a residual variance model to the data. Residual variance models based on small samples are not very robust, which can easily be seen if the data are jackknifed or bootstrapped. One way to overcome this is to assume a common residual variance model for all observations, instead of a residual variance model for each subject. This assumption is not such a leap of faith. For GLS, first fit each subject and then pool the residuals. Use the pooled residuals to estimate the residual variance model parameters and then iterate in this manner until convergence. For ELS, things are a bit trickier but are still doable. 

A Monte Carlo experiment was conducted comparing the different algorithms. Dose-effect data were simulated using an Emax model with intercept where E0 was set equal to 200, Emax was set equal to -100, ED50 was set equal to 25, and dose was fixed at the following levels 0, 10, 20, 40, 80, 160, and 320. A total of 25 subjects were simulated per dose level. The only source of error in this simulation was random error, which was added to each observation under the following residual variance models ... 

Table 4.3 Results of Monte Carlo simulation testing the effect of residual variance model misspecification on nonlinear regression parameter estimates.

True residual variance model Additive with constant CV (15%) ... 

True residual variance model Mixture distribution Mean value Between simulation CV(%) of parameter estimates ... 

Transform-Both-Sides Approach with Accompanying Residual Variance Model... 

Carroll and Ruppert (1988) point out that even though a transformation may transform a distribution to normality, there is no guarantee that the transform will lead to homoscedasticity. Hence one may need to find a suitable transformation and a suitable residual variance model. This leads to the model... 

For the OLS model, a 4-compartment model could not even be estimated. The parameter estimate for Q4 kept reaching its lower bound and eventually became zero. Hence for this residual variance model, a 3-com-partment model was fit. The goodness of fit plot showed that at later time points the model began to significantly underpredict concentrations. When inverse observed or... 

Legend NA, for the OLS residual variance model, the parameters associated with the 4th compartment (Q4, V4) could not be estimated. At most three compartments could be identified. The symbol denotes that 95% confidence interval contains zero NA, could not be estimated (see text for details). 

Residual variance models were described in detail in the chapter on variance models and weighting. What was detailed in that chapter readily extends to nonlinear mixed effects models. As might be expected, residual variance models model the random, unexplained variability in the regression function f. Hence, the structural model is extended to... 

---