Residual variance model likelihood

Equation (4.2) is called a residual variance model, but it is not a very general one. In this case, the model states that random, unexplained variability is a constant. Two methods are usually used to estimate 0 least-squares (LS) and maximum likelihood (ML). In the case where e N(0, a2), the LS estimates are equivalent to the ML estimates. This chapter will deal with the case for more general variance models when a constant variance does not apply. Unfortunately, most of the statistical literature deals with estimation and model selection theory for the structural model and there is far less theory regarding choice and model selection for residual variance models. 

RESIDUAL VARIANCE MODEL PARAMETER ESTIMATION USING MAXIMUM LIKELIHOOD... 

An independent method to identify the stochastic errors of impedance data is described in Chapter 21. An alternative approach has been to use the method of maximum likelihood, in which the regression procedure is used to obtain a joint estimate for the parameter vector P and the error structure of the data. The maximum likelihood method is recommended under conditions where the error structure is unknown, but the error structure obtained by simultaneous regression is severely constrained by the assumed form of the error-variance model. In addition, the assumption that the error variance model can be obtained by minimizing the objective function ignores the differences eimong the contributions to the residual errors shown in Chapter 21. Finedly, the use of the regression procedure to estimate the standard deviation of the data precludes use of the statistic... 

Goodness-of-Fit. It is implied in steps 2 to 6 above that diagnostic plots (e.g., weighted residual versus time, weighted residual versus predicted observations, population observed versus predicted concentrations, individual observed versus predicted concentrations) and a test statistic such as the likelihood ratio test would be used in arriving at the base model (see Section 8.6.1.1 for goodness of fit). Once the base model (with optimized structural and variance models) has been obtained, the next step in the PM model identification process is the development of the population model. 

The AIC used by Yamaoka et al. is not the exact AIC reported by Akaike but is slightly different. However, its use is valid if all the models under consideration all have the same log-likelihood function. Yamaoka et al. (1978) demonstrated the use of the AIC by distinguishing between mono-exponential, bi-exponential, and tri-exponential equations. However, not all software packages use this definition. For example, WinNonlin (Pharsight Corp., Cary NC) uses Eq. (1.45) with p equal to only the number of structural model parameters and does not include the estimated residual variance. SAAM II (SAAM Institute, Seattle, WA) calculates the AIC as... 

Both Astrom (53) and Box and Jenkins (54) have developed modeling approaches for equation (13), which involve obtaining maximum likelihood estimates of the parameters in the postulated model followed by diagnostic checking of the sum of the residuals. The Box and Jenkins method also develops a detailed model for the process disturbance. Both of the above references include derivations of the minimum variance control. 

Liao (2000) derived a test statistic for single dispersion effects in 2" k designs. He applied the generalized likelihood ratio test for a normal model to the residuals after fitting a location model, which results in Bartlett s (1937) classical test for comparing variances in one-way layouts. The test is then applied, in turn, to compare the variances at the two levels of each of the k experimental factors. We caution that the test statistic (equation (3) in Liao) is written incorrectly. 

Verbyla, A. P. (1993). Modelling variance heterogeneity Residual maximum likelihood and diagnostics. Journal of the Royal Statistical Society B, 55, 493-508. 

NONMEM is a one-stage analysis that simultaneously estimates mean parameters, fixed-effect parameters, interindividual variability, and residual random effects. The fitting routine makes use of the EES method. A global measure of goodness of fit is provided by the objective function value based on the final parameter estimates, which, in the case of NONMEM, is minus twice the log likelihood of the data (1). Any improvement in the model would be reflected by a decrease in the objective function. The purpose of adding independent variables to the model, such as CLqr in Equation 10.7, is usually to explain kinetic differences between individuals. This means that such differences were not explained by the model prior to adding the variable and were part of random interindividual variability. Therefore, inclusion of additional variables in the model is warranted only if it is accompanied by a decrease in the estimates of the intersubject variance and, under certain circumstances, the intrasubject variance. 

Carroll and Ruppert (1988) and Davidian and Gil-tinan (1995) present comprehensive overviews of parameter estimation in the face of heteroscedasticity. In general, three methods are used to provide precise, unbiased parameter estimates weighted least-squares (WLS), maximum likelihood, and data and/or model transformations. Johnston (1972) has shown that as the departure from constant variance increases, the benefit from using methods that deal with heteroscedasticity increases. The difficulty in using WLS or variations of WLS is that additional burdens on the model are made in that the method makes the additional assumption that the variance of the observations is either known or can be estimated. In WLS, the goal is not to minimize the OLS objective function, i.e., the residual sum of squares,... 

---