Multiple regression, forward

How do we set about variable selection One obvious approach is to examine the pair-wise correlations between the response and the physicochemical descriptors. One form of model building, forward stepping multiple regression, begins by choosing the descriptor that has the highest correlation with a response variable. If the response is a categorical variable such as toxic/non-toxic,... 

The data were analysed in several stages. Descriptive statistics and bivariate correlations were calculated for independent, dependent and control variables. Control variables with significant bivariate correlations with outcome as measured by the NBAS scales were used in forward stepwise multiple regression analyses to determine the best joint predictors of the NBAS. After the best multiple regression model was constructed from the non-lead variables, lead measurements recorded at the three time points were added to the model to determine the relationship between each of the lead measurements and the adjusted NBAS scores. The overall plan of analysis follows Bellinger et al (1984 this volume). Analyses were performed with SAS programs. 

Figures 11 and 12 illustrate the performance of the pR2 compared with several of the currently popular criteria on a specific data set resulting from one of the drug hunting projects at Eli Lilly. This data set has IC50 values for 1289 molecules. There were 2317 descriptors (or covariates) and a multiple linear regression model was used with forward variable selection the linear model was trained on half the data (selected at random) and evaluated on the other (hold-out) half. The root mean squared error of prediction (RMSE) for the test hold-out set is minimized when the model has 21 parameters. Figure 11 shows the model size chosen by several criteria applied to the training set in a forward selection for example, the pR2 chose 22 descriptors, the Bayesian Information Criterion chose 49, Leave One Out cross-validation chose 308, the adjusted R2 chose 435, and the Akaike Information Criterion chose 512 descriptors in the model. Although the pR2 criterion selected considerably fewer descriptors than the other methods, it had the best prediction performance. Also, only pR2 and BIC had better prediction on the test data set than the null model.

Alternatively, instead of using the EBE of the parameter of interest as the dependent variable, an estimate of the random effect (t ) can be used as the dependent variable, similar to how partial residuals are used in stepwise linear regression. Early population pharmacokinetic methodology advocated multiple linear regression using either forward, backwards, or stepwise models. A modification of this is to use multiple simple linear models, one for each covariate. For categorical covariates, analysis of variance is used instead. If the p-value for the omnibus F-test or p-value for the T-test is less than some cut-off value, usually 0.05, the covariate is moved forward for further examination. Many reports in the literature use this approach. 

Forward or backward elimination (as used in multiple linear regression)... 

Stepwise multiple linear regression. This is a modified form of forward selection. The model starts out including only one variable, and more variables are subsequently added. But at each stage a BE-style test is also applied. If a variable is added, but becomes less important as a result of subsequent additions, SMLR will allow its removal from the model. 

Ilari et al. [85] used scatter correction to determine the particle sizes of materials. Particle sizes of both organic and inorganic materials were determined by this technique. O Neil et al. [86] measured the cumulative particle size distribution of microcrystalline cellulose with diffuse reflectance NIR. Both multiple linear regression (MLR) and PCA were used for the work. The results were consistent with those obtained by forward-angle laser light scattering. Rantanen and Yliruusi [87] predicted... 

Chapter 4 retrieves the basic ideas of classical univariate calibration as the standpoint from which the natural and intuitive extension of multiple linear regression (MLR), arises. Unfortunately, this generalization is not suited to many laboratory tasks and, therefore, the problems associated with its use are explained in some detail. Such problems justify the use of other more advanced techniques. The explanation of what the multivariate space looks like and how principal components analysis can tackle it is the next step forward. This constitutes the root of the regression methodology presented in the following chapter. 

Backward elimination is a variable selection algorithm for multiple linear regression it starts with all variables in the model and eliminates all nonsignificant variables see forward selection as well. 

Method of Solution The Marquardt method using the Gauss-Newton technique, described in Sec. 7.4.4, and the concept of multiple nonlinear regression, covered in Sec. 7.4.5, have been combined together to solve this example. Numerical differentiation by forward finite differences is used to evaluate the Jacobian matrix defined by Eq. (7.164). 

---