Model misspecification

Model development is intimately linked to correctly assigning model parameters to avoid problems of identifiability and model misspecification [27-29], A full understanding of the objectives of the modeling exercise, combined with carefully planned study protocols, will limit errors in model identification. Compartmental models, as much as any other modeling technique, have been associated with overzealous interpretation of the model and parameters. 

The statistical submodel characterizes the pharmacokinetic variability of the mAb and includes the influence of random - that is, not quantifiable or uncontrollable factors. If multiple doses of the antibody are administered, then three hierarchical components of random variability can be defined inter-individual variability inter-occasional variability and residual variability. Inter-individual variability quantifies the unexplained difference of the pharmacokinetic parameters between individuals. If data are available from different administrations to one patient, inter-occasional variability can be estimated as random variation of a pharmacokinetic parameter (for example, CL) between the different administration periods. For mAbs, this was first introduced in sibrotuzumab data analysis. In order to individualize therapy based on concentration measurements, it is a prerequisite that inter-occasional variability (variability within one patient at multiple administrations) is lower than inter-individual variability (variability between patients). Residual variability accounts for model misspecification, errors in documentation of the dosage regimen or blood sampling time points, assay variability, and other sources of error. 

Then, given a model for data from a specific drug in a sample from a population, mixed-effect modeling produces estimates for the complete statistical distribution of the pharmacokinetic-dynamic parameters in the population. Especially, the variance in the pharmacokinetic-dynamic parameter distributions is a measure of the extent of inherent interindividual variability for the particular drug in that population (adults, neonates, etc.). The distribution of residual errors in the observations, with respect to the mean pharmacokinetic or pharmacodynamic model, reflects measurement or assay error, model misspecification, and, more rarely, temporal dependence of the parameters. 

Population pharmacokinetic parameters quantify population mean kineticS/ between-subject variability (intersubject variability)/ and residual variability. Residual variability includes within-subject variability/ model misspecification/ and measurement error. This information is necessary to design a dosage regimen for a drug. If all patients were identical/ the same dose would be appropriate for all. However/ since... 

Optimize the Structural Model It is important to ensure that the structural model describes the underlying patterns in the data. If the data speaks to the existence of two clearly distinct absorption profiles, then a mixture model should be tested to ensure that the absorption phase of the profile is well characterized. If absorption, for example, could be better characterized with sequential first-order absorption models instead of a simple first-order model, the appropriate model should be used to eliminate bias due to model misspecification. 

Partial residuals are produced with GAM, and not the usual residual plots. Plots of residuals and functions of residuals are useful particularly for identifying patterns in the data that may suggest heterogeneity of variance or bias due to deterministic model misspecification or misspecifications of the regression variables. One particular form of bias that may exist occurs when a predictor variable is included in the model in a linear form when it actually has a curvilinear or nonlinear relationship with the response variable. A plot used by Ezekiel (23) and later referred to as a partial residual plot by Larsen and McCleary (24) is useful for this purpose. Partial residuals are defined as... 

The above argument could be extended to say that, because the reality is highly complex, any model will likely underestimate Cmax- This may be true, but the extent should be taken in perspective. The traditional use of measured Cmax also underestimates Cmax, because it is unlikely that Tmax is among the sampling times. A prerequisite of using modeling in any circumstance should be that the influence of potential model misspecification is limited, compared with alternative choices. 

Because of its many assumptions, a population model, especially with all the pre-specification demanded in this framework, is unlikely to be true. However, one can argue that this framework exerts the influence of model misspecification primarily on study power. This is because a misspecified model would generally result in lower power although not larger Type I error. In addition, this approach maintains a more realistic confidence interval width instead of an overly optimistic (short) one. By maximizing the model as much as data can be expected to support, the impact on Type I error is minimized. Therefore, the hypothesis test is made as conservative as possible, and thus suitable for BE assessment. 

The geometric means step in the above two methods is appropriate when the error in response measurements (including model misspecification error) is lognormal distributed. The geometric means step can be replaced by arithmetic means when normally distributed errors are more appropriate. From a regulatory perspective, this choice should be made before data collection. 

We used Monte Carlo simulation to gain insight of the performances of the designs and the analysis methods. As initial explorations, the simulations focused on the accuracy and precision of estimating relative bioavailabihty. For potential regulatory applications, it is important to study the effect of error model (mis)specification and the influence of structural model misspecification. Of particular interest are the following issues ... 

For scenarios I-III and VI, the study design was chosen as using the original doses of 0,1, and 4 puffs. Scenario IV and V aimed at studying the effect of model misspecification, that is, when the data do not allow accurate estimation of the E model. Because edso was 2.99 puffs in the data generation model, the study design using doses of 0,1, and 2 puffs as chosen for scenarios IV and V. 

For the last topic in this section, the situation where we have a skewed modal eta distribution is revisited, but this time there is no model misspecification. Using techniques from Section 28.9, simulate and then estimate rich PK data as in the first example (100 subjects, 2400 observations), such that 80% of the population has a Ka of 1 and the remaining 20% of the population has a Ka of 4. Both subpopulations have interindividual variability in Ka consistent with a CV of 47%. Using a two PROBLEM control stream (c2i. txt) and data skeleton (datas. txt), first simulate the data and record as part of the data which subpopulation each patient is assigned to with the following code ... 

Many of the plots just suggested are not limited to ordinary residuals. Weighted residuals, partial residuals, studentized residuals, and others can all be used to aid in model diagnostics. Beyond residual plots, other plots are also informative and can help in detecting model inadequacies. One notable plot is a scatter plot of observed versus predicted values usually with the line of unity overlaid on the plot (Fig. 1.8). The model should show random variation around the line of unity. Systematic deviations from the line indicate model misspecification whereas if the variance of the predicted values increases as the observed values increase then the variance model may be inappropriate. 

ELS and that earlier enthusiasm expressed for ELS must be tempered. Two other notable observations were made. First, if the residual variance model was incorrectly specified with ELS, this adversely affected the ELS parameter estimates. Belanger et al. (1971) indicated that this should not be a problem with GLS. Second, considerable advantage was gained when was treated as an estimable parameter, rather than a fixed value as in IRWLS or WLS, as estimable residual variance models are more robust to residual variance model misspecification than fixed residual variance models. 

Table 4.3 Results of Monte Carlo simulation testing the effect of residual variance model misspecification on nonlinear regression parameter estimates.

Figure 6.1 illustrates this relationship graphically for a 1-compartment open model plotted on a semi-log scale for two different subjects. Equation (6.15) has fixed effects (3 and random effects U . Note that if z = 0, then Eq. (6.15) simplifies to a general linear model. If there are no fixed effects in the model and all model parameters are allowed to vary across subjects, then Eq. (6.16) is referred to as a random coefficients model. It is assumed that U is normally distributed with mean 0 and variance G (which assesses between-subject variability), s is normally distributed with mean 0 and variance R (which assesses residual variability), and that the random effects and residuals are independent. Sometimes R is referred to as within-subject or intrasubject variability but this is not technically correct because within-subject variability is but one component of residual variability. There may be other sources of variability in R, sometimes many others, like model misspecification or measurement variability. However, in this book within-subject variability and residual variability will be used interchangeably. Notice that the model assumes that each subject follows a linear regression model where some parameters are population-specific and others are subject-specific. Also note that the residual errors are within-subject errors. 

Additionally, many sources of variability, such as model misspecification, or dosing and sampling history, may lead to residual errors that are time dependent. For example, the residual variance may be larger in the absorption phase than in the elimination phase of a drug. Hence, it may be necessary to include time in the residual variance model. One can use a more general residual variance model where time is explicitly taken into account or one can use a threshold model where one residual variance model accounts for the residual variability up to time t, but another model applies thereafter. Such models have been shown to result in significant model improvements (Karlsson, Beal, and Sheiner, 1995). 

Box (1976), in one of the most famous quotes reported in the pharmacokinetic literature, stated all models are wrong, some are useful. This adage is well accepted. The question then becomes how precise or of what value are the parameter estimates if the model or the model assumptions are wrong. A variety of simulation studies have indicated that population parameter estimates are surprisingly robust to all kinds of misspe-cifications, but that the variance components are far more sensitive and are often very biased when misspeci-fication of the model or when violations of the model assumptions occur. Some of the more conclusive studies examining the effect of model misspecification or model assumption violations on parameter estimation will now be discussed. 

Wade et al. (1993) simulated concentration data for 100 subjects under a one-compartment steady-state model using either first-or zero-order absorption. Simulated data were then fit using FO-approximation with a first-order absorption model having ka fixed to 0.25-, 0.5-, 1-, 2-, 3-, and 4 times the true ka value. Whatever value ka was fixed equal to, clearance was consistently biased, but was relatively robust with underpredictions of the true value by less than 5% on average. In contrast, volume of distribution was very sensitive to absorption misspecification, but only when there were samples collected in the absorption phase. When there were no concentration data in the absorption phase, significant parameter bias was not observed for any parameter. The variance components were far more sensitive to model misspecification than the parameter estimates with some... 

A true PPC requires sampling from the posterior distribution of the fixed and random effects in the model, which is typically not known. A complete solution then usually requires Markov Chain Monte Carlo simulation, which is not easy to implement. Luckily for the analyst, Yano, Sheiner, and Beal (2001) showed that complete implementation of the algorithm does not appear to be necessary since fixing the values of the model parameters to their final values obtained using maximum likelihood resulted in PPC distributions that were as good as the full-blown Bayesian PPC distributions. In other words, using a predictive check resulted in distributions that were similar to PPC distributions. Unfortunately they also showed that the PPC is very conservative and not very powerful at detecting model misspecification. 

Intuitively, parametric models tend to yield estimates that are less variable than nonparametric and semiparametric models. However, parametric models are more sensitive to model misspecification. What model misspecification means is that if the distribution of the data does not match the model assumed, then the answers gotten from the model will tend to be quite wrong or, in statistical jargon, biased. By contrast, nonparametric and semiparametric models tend to make fewer... 

Jarvis, C. B., MacKenzle, S. B., and PodsakofF, P. M. (2003), "A critical review of construct indicators and measurement model misspecification in marketing and consumer research," Journal of Consumer Research, 30 (2), 199-218. 

---