Covariates selection

Arena VC, Sussman NB, Mazumdar S, et al. The utility of structure-activity relationship (SAR) models for prediction and covariate selection in developmental toxicity Comparative analysis of logistic regression and decision tree models. SAR QSAR Environ Res. 2004 15(1) 1-18. 

Step 6. With the appropriate pharmacostatistical models, population model building is performed using covariates retained in step 5 with the covariate selection level set at a= 0.005. The backward elimination for covariate selection in applied to each of the 100 bootstrap samples. The covariates found to be important in explaining the variablilty in the parameter of interest are used to build the final population PM model. 

Most applications of PopPK methodology follow this general scheme but they are all flawed in the sense that they suffer from the same problems as are seen with stepwise regression, such as selection bias, overstated importance of retained covariates, and possibly invalid distributional assumptions (Wahlby, Jonsson, and Karlsson, 2002). However, it must be stressed that there is no universally accepted method for covariate selection in any regression-based model. 

There have only been a few studies comparing the different covariate selection methods with most using Monte Carlo comparisons where the true data generating model is known. It should be stressed at the outset that no method is universally superior to the others— they all sometimes choose the wrong covariates or miss important covariates. Most often the choice of selection method is a personal one dependent on the analysis and analyst. 

In a study of a pharmacodynamic model of HIV viral load (HIV-1 RNA copies) over time with 31 cov-ariates, the three different methods identified three different sets of important covariates. The NL-based method produced a set of covariates with the smallest number, whereas the LRT produced the largest set of important covariates. They concluded that EBE-based methods are the most reliable for covariate selection. They also found that nonparametric regression methods were more likely to select fewer important covariates than parametric regression methods. The authors then used Monte Carlo simulation to examine the power and Type I error rate of the methods. 

They concluded that the Type I error rate for EBE-based methods were near the nominal level (a = 0.05) under most cases. The Type I errors for the NL-based methods were near the nominal level in most cases, but were smaller under sparse data conditions and with small sample sizes. The LRT cases consistently inflated the Type I error rate and that, not surprisingly, the LRT was the most powerful of the methods examined. This latter result can be rationalized as thinking that the inflated Type I error rate acts as a constant to inflate statistical power at nonzero effect sizes. They concluded that the LRT was too liberal for sparse data, while the NL-based methods were too conservative, and that the EBE-based methods were the most reliable for covariate selection. 

Gobburu, J. and Lawrence, J. Application of resampling techniques to estimate exact significance levels for covariate selection during nonlinear mixed effects model building Some inferences. Pharmaceutical Research 2002 19 92-98. 

Covariate selection Examine covariate addition to model using generalized additive modeling (GAM) or equivalent stepwise insertion techniques. 

NONMEM itself can be utilized to perform covariate selection without GAM. In this setting, the stractural model is utilized as the base model and single covariates are added in any manner and examined for their effect on the parameter of interest via a drop in the objective function. Each covariate is ranked against the base model by the change in OF, and ranked covariates (highest to lowest) are introduced onto the base model in a stepwise manner with only significant covariates added. Upon completion of the fully saturated model, a backward deletion procedure can be utilized to remove redundant covariates. 

As with other approaches, the propensity score model is only as good as the covariates selected to provide the adjustment. A propensity score is a single metric that is intended to account for all of the explanatory variables that predict who will receive treatment. Propensity scores generally balance observed confounders but do not necessarily produce balance on factors not incorporated into the model. Such imbalances represent a particular problem... 

For all three spectral windows trialed there was no significant difference in the classification accuracy, sensitivity or specificity when the number of covariant selected IR absorbance values was reduced from 36 to 16 points. This indicates that the spectra contain some redundant information, which is not surprising given that the 6cm spectral resolution employed for spectral acquisition is small when compared with the inherent bandwidth of the features... 

Spectral window ( cmCf Number of covariance selected input feature values Classification Sensitivity (%) Specificity (%) accuracy (%) ... 

Price GR (1972) Extensitm of covariance selection mathematics. Ann Hiun Genet... 

Multiple regression was used with entry of a set of covariates selected a priori on the basis of a plausible relationship with preschool intelligence. These were sex, age at testing, race, birth order, maternal PPVT-R, parental education, maternal attitude towards child-rearing, and, for ages 1, 2 and 3 years, the HOME Inventory. Alpha was set at 0.01 to minimize study wise error but testwise p values for each analysis are provided. 

---