Constant variance assumption

It should be emphasized that if the concentration range is greater than one order of magnitude (long-range calibration), violations of the constant variance assumption of OLS are frequent. Especially, homoscedasticity is rarely found hence, a more general model needs to be applied. 

We will focus in our development in this section on the unpaired t-test. The constant variance assumption can be assessed by undertaking a test (the so-called F-test) relating to the hypotheses ... 

The constant variance assumption can be relaxed via either a rescaling of the response or a weighted fit (4). Similarly, if an appropriate model is used, the normality assumption may be relaxed (4). For example, with a dichotomous response, a logit-log model may be appropriate (5). Other response patterns (e.g., Poisson) may be fit via a generalized linear model (6). For quantitative responses, it is often most practical to find a rescaling or transformation of the response scale to achieve nearly constant variance and nearly normal responses. Finally, if samples are grouped, then blocks or other experiment design structures must be included in the model (7-12). 

In general terms, the violation of these assumptions is accommodated via the modeling of variance in the response and across individuals and through the definition of correlation among variance partitions. For example, the constant variance assumption in Table 15.3 is handled via the specification of the intraindividual error function ( SIGMA block in NONMEM nomenclature) that incorporates potential... 

The essential assumption of this manuscript is the existence of a constant variance of Gaussian errors along the trajectory. While we attempted to correlate the variance with the high frequency motions, many uncertainties and questions remain. These are topics for future research. 

Even if we make the stringent assumption that errors in the measurement of each variable ( >,. , M.2,...,N, j=l,2,...,R) are independently and identically distributed (i.i.d.) normally with zero mean and constant variance, it is rather difficult to establish the exact distribution of the error term e, in Equation 2.35. This is particularly true when the expression is highly nonlinear. For example, this situation arises in the estimation of parameters for nonlinear thermodynamic models and in the treatment of potentiometric titration data (Sutton and MacGregor. 1977 Sachs. 1976 Englezos et al., 1990a, 1990b). 

Now, if (m2 > g), the solution of Eq. (10.24), under the assumption of an independent and normal error distribution with constant variance can be obtained as the maximum likelihood estimator of d and is given by... 

In transforming the independent variables alone, it is assumed that the dependent variable already has all the properties desired of it. For example, if the /s are normally and independently distributed with constant variance, at least approximately, then any transformations such as described in Section VI,B,1 would be unnecessary. Under such assumptions, Box and Tidwell (B17) have shown how to transform the independent variables to reduce a fitted linear function to its simplest form. For example, a function that has been empirically fitted by... 

Both assumptions are mainly needed for constructing confidence intervals and tests for the regression parameters, as well as for prediction intervals for new observations in x. The assumption of normal distribution additionally helps avoid skewness and outliers, mean 0 guarantees a linear relationship. The constant variance, also called homoscedasticity, is also needed for inference (confidence intervals and tests). This assumption would be violated if the variance of y (which is equal to the residual variance a2, see below) is dependent on the value of x, a situation called heteroscedasticity, see Figure 4.8. 

The least-squares curve-of-best-fit procedure implicitly assumes the same variance (standard deviation) at all concentrations. This assumption is rarely correct. Figure 3a shows hypothetical replicate standard analysis data with constant variance. This pattern is almost never seen in routine chemical analyses. Figure 3b shows a much more realistic pattern in which the variance increases with concentration. 

Since data from chromatography standards typically do not satisfy the assumptions of constant variance nor linearity, a procedure described above for fitting a family of transformations on the y and x. will be used We assume for the rest of... 

To model the relationship between PLA and PLR, we used each of these in ordinary least squares (OLS) multiple regression to explore the relationship between the dependent variables Mean PLR or Mean PLA and the independent variables (Berry and Feldman, 1985).OLS regression was used because data satisfied OLS assumptions for the model as the best linear unbiased estimator (BLUE). Distribution of errors (residuals) is normal, they are uncorrelated with each other, and homoscedastic (constant variance among residuals), with the mean of 0. We also analyzed predicted values plotted against residuals, as they are a better indicator of non-normality in aggregated data, and found them also to be homoscedastic and independent of one other. 

When a model is fitted to experimental data, there will always be deviations between the observed responses and the responses calculated fi om the model. With an adequate model, these deviations should be nothing but a manifestation of the experimental enor. Under these circumstances, it can be assumed that the residuals should also be normally and independently distributed and have a constant variance. Therefore, any indications that these assumptions are violated should jeopardize the model. 

Indications of this can be obtained by plotting tbe residuals in different ways, to make sure that th do not show any abnormal pattern. Sometimes it is found that the plot of the residuals against the predicted response has a funnel-like pattern, which shows that the size of tbe residuals are dependent on the magnitude of tbe responses. A situation when this is often encountered is when integrated chromatographic peaks are used to determine concentrations in tbe sample. When such patterns are observed it is a clear indication that the assumption of a constant variance is violated. This obstacle can sometimes be removed by a mathematical transformation of tbe response variable y... 

Autocorrelation in data affects the accuracy of the charts developed based on the iid assumption. One way to reduce the impact of autocorrelation is to estimate the value of the observation from a model and compute the error between the measured and estimated values. The errors, also called residuals, are assumed to have a Normal distribution with zero mean. Consequently regular SPM charts such as Shewhart or CUSUM charts could be used on the residuals to monitor process behavior. This method relies on the existence of a process model that can predict the observations at each sampling time. Various techniques for empirical model development are presented in Chapter 4. The most popular modeling technique for SPM has been time series models [1, 202] outlined in Section 4.4, because they have been used extensively in the statistics community, but in reality any dynamic model could be used to estimate the observations. If a good process model is available, the prediction errors (residual) e k) = y k)—y k) can be used to monitor the process status. If the model provides accurate predictions, the residuals have a Normal distribution and are independently distributed with mean zero and constant variance (equal to the prediction error variance). 

When reliable O are available, the adjustable parameters may be obtained from Equation (3.27), but the derivation is more involved than the simpler case of uniform variance. More information regarding the full derivation is provided by Press et al. [10]. The remainder of this section focuses on regression analyses with the assumption of constant variance, with separate discussions of simple linear, generalized multiple linear, and nonlinear regression. 

Under the assumption that the residuals are independent, normally distributed with mean 0 and constant variance, when the sample size is large, standardized residuals greater than 2 are often identified as suspect observations. Since asymptotically standardized residuals are normally distributed, one might think that they are bounded by — oo and +00, but in fact, a stand-ardized residual can never exceed y/(n - p)(n - l)n-1 (Gray and Woodall, 1994). For a simple linear model with 19 observations, it is impossible for any standardized residual to exceed 4. Standardized residuals suffer from the fact that they a prone to ballooning in which extreme cases of x tend to have smaller residuals than cases of x near the centroid of the data. To account for this, a more commonly used statistic, called studentized or internally studentized residuals, was developed... 

Notice that two assumptions have been made normality of the responses and constant variance. The result is that the conditional distribution itself is normally distributed with mean 0O + (fix and variance joint distribution function at any level of X can be sliced and still have a normal distribution. Also, any conditional probability distribution function of Y has the same standard deviation after scaling the resulting probability distribution function to have an area of 1. 

With linear models, exact inferential procedures are available for any sample size. The reason is that as a result of the linearity of the model parameters, the parameter estimates are unbiased with minimum variance when the assumption of independent, normally distributed residuals with constant variance holds. The same is not true with nonlinear models because even if the residuals assumption is true, the parameter estimates do not necessarily have minimum variance or are unbiased. Thus, inferences about the model parameter estimates are usually based on large sample sizes because the properties of these estimators are asymptotic, i.e., are true as n —> oo. Thus, when n is large and the residuals assumption is true, only then will nonlinear regression parameter estimates have estimates that are normally distributed and almost unbiased with minimum variance. As n increases, the degree of unbiasedness and estimation variability will increase. 

When dealing with pharmacokinetic data, it is actually quite rare for the assumption of constant variance to be met. When the observations exhibit nonconstant variance, the data are heteroscedastic and the basic model needs to be modified to... 

Notice that nothing beyond the first two moments of Y is being assumed, i.e., only the mean and variance of the data are being defined and no distributional assumptions, such as normality, are being made. In residual variance model estimation, the goal is to understand the variance structure as a function of a set of predictors, which may not necessarily be the same as the set of predictors in the structural model (Davidian and Car-roll, 1987). Common, heteroscedastic error models are shown in Table 4.1. Under all these models, generic s is assumed to be independent, having zero mean and constant variance. 

One method for dealing with heteroscedastic data is to ignore the variability in Y and use unweighted OLS estimates of 0. Consider the data shown in Fig. 4.2 having a constant variance plus proportional error model. The true values were volume of distribution = 10 L, clearance = 1.5 L/h, and absorption rate constant = 0.7 per/h. The OLS estimates from fitting a 1-compartment model to the data were as follows volume of distribution = 10.3 0.15L, clearance = 1.49 0.01 L/h, and absorption rate constant =0.75 0.03 per h. The parameter estimates themselves were quite well estimated, despite the fact that the assumption of constant variance was violated. Figure 4.3 presents the residual plots discussed in the previous section. The top plot, raw residuals versus predicted values, shows that as the predicted values increase so do the variance of the residuals. This is confirmed by the bottom two plots of Fig. 4.3 which indicate that both the range of the absolute value of the residuals and squared residuals increase as the predicted values increase. 

In contrast, the data in the top plot of Fig. 4.2 using a constant residual variance model led to the following parameter estimates after fitting the same model volume of distribution =10.2 0.10L, clearance = 1.49 0.008 L/h, and absorption rate constant = 0.71 0.02 per h. Note that this model is the data generating model with no regression assumption violations. The residual plots from this analysis are shown in Fig. 4.4. None of the residual plots show any trend or increasing variance with increasing predicted value. Notice that the parameter estimates are less biased and have smaller standard errors than the estimates obtained from the constant variance plus proportional error model. 

Carroll and Ruppert (1988) and Davidian and Gil-tinan (1995) present comprehensive overviews of parameter estimation in the face of heteroscedasticity. In general, three methods are used to provide precise, unbiased parameter estimates weighted least-squares (WLS), maximum likelihood, and data and/or model transformations. Johnston (1972) has shown that as the departure from constant variance increases, the benefit from using methods that deal with heteroscedasticity increases. The difficulty in using WLS or variations of WLS is that additional burdens on the model are made in that the method makes the additional assumption that the variance of the observations is either known or can be estimated. In WLS, the goal is not to minimize the OLS objective function, i.e., the residual sum of squares,... 

In the model presented above, the R matrix elements, or matrix of within-subject errors, are uncorrelated and of constant variance across all subjects. When this is the case, Eq. (6.15) is sometimes called the conditional independence model since it assumes that responses for the ith subject are independent of and conditional on the U s and (3. At times, however, this may be an unrealistic assumption since it seems more likely that observations within a subject are correlated. For example, if the model were misspecified, then parts of the data over time would be more correlated than other parts (Karls-son, Beal, and Sheiner, 1995). Hence, it is more realistic to allow the within-subject errors to be correlated. 

It is assumed that the residuals are independent, normally distributed, with mean zero and constant variance. These are standard assumptions for maximum likelihood estimation and can be tested using standard methods examination of histograms, autocorrelation plots (ith residual versus lag-1 residual), univariate analysis with a test for normality, etc. 

Mean and variance are considered to be constant. This assumption only holds in the case of a stationary process. 

---