All subset regression

Neither all subsets regression nor the Hamada-Wu stepwise algorithm represents a complete solution. All subsets regression provides a complete search but ignores effect heredity. The Hamada-Wu approach identifies models obeying effect heredity but has an incomplete search and may miss the best effect heredity models. 

The Bayesian approach is more than a tool for adjusting the results of the all subsets regression by adding appropriate effects to achieve effect heredity. Take, for example, the sixth model in Table 4 which consists of Al,Bl, AlDq, BlHl, BlHq, BqHq. The AlDq effect identified as part of this model does not appear in the best subsets of size 1-6 in Table 3. The Bayesian procedure has therefore discovered an additional possible subset of effects that describes the data. 

All subsets regression was used for the development of the models. The criteria used for defining the best model were and Mallow s For each of the properties examined, initial models used only the TIs and HBi as potential variables. Subsequently, we added the three geometric variables to examine the improvement provided by the addition of geometric information. 

With the 16 TIs, all subsets regression resulted in a seven-parameter model as follows ... 

Using all subsets regression with the selected TIs and HBi as independent variables resulted in a nine-parameter model ... 

All three of these methods that we use for variable selection are prone to entrapment in local minima, i.e., they find a combination of variables that cannot be improved on in the next step (removal or addition of one variable) for the criterion function, which can be avoided by performing either a Tabu search (TS) or the more computationally expensive all subsets regression. We discuss the second of these two methods in the next section and refer readers to the papers by Glover for details of the TS method. 

To carry out an all subset regression means that all models will be constructed of size one (of which there are 30), of size two (there are... 

This classification method can be considered a qualitative equivalent to the subset regression strategy for improving quantitative models, discussed earlier in Section 8.3.8.5. Instead of defining a common sample space for all classes and then determining class membership based on the location of the unknown sample in the common space, SIMCA actually defines a different space for each class. For this discussion, the parameter Z will be used to denote the number of known classes in the calibration data. 

Abraham et al. (1999) studied forward selection and all subsets selection in detail. They showed, by simulating data from several different experiments, that the factors identified as active could change completely if a different fraction was used and that neither of these methods could reliably find three factors which have large effects. However, they concluded that all subsets selection is better than forward selection. Kelly and Voelkel (2000) showed more generally that the probabilities of type-II errors from stepwise regression are high. 

We are particularly interested in finding, for a given kpredicting functions (best subset selection, BSS). The trivial solution of this problem is to search all fc-subsets, determine a predicting function for each, and then select the best of these. However, this requires a high computational effort. There are Unear algebra techniques that can be used to minimize the effort in the case of linear regression [82]. Nevertheless, often it is impossible to search all subsets in reasonable time. 

Several approaches have been investigated recently to achieve this multivariate calibration transfer. All of these require that a small set of transfer samples is measured on all instruments involved. Usually, this is a small subset of the larger calibration set that has been measured on the parent instrument A. Let Z indicate the set of spectra for the transfer set, X the full set of spectra measured on the parent instrument and a suffix Aor B the instrument on which the spectra were obtained. The oldest approach to the calibration transfer problem is to apply the calibration model, b, developed for the parent instrument A using a large calibration set (X ), to the spectra of the transfer set obtained on each instrument, i.e. and Zg. One then regresses the predictions (=Z b ) obtained for the parent instrument on those for the child instrument yg (=Z b ), giving... 

Thus we see that we cannot arbitrarily select any subset of the data to use in our computations it is critical to keep all the data, in order to achieve the correct result, and that requires using the regression approach, as we discussed above. If we do that, then we find that the correct fitting equation is (again, this system of equations is simple enough to do for practice - the matrix inversion can be performed using the row operations as we described previously) ... 

This method can be considered a calibration transfer method that involves a simple instrument-specific postprocessing of the calibration model outputs [108,113]. It requires the analysis of a subset of the calibration standards on the master and all of the slave instmments. A multivariate calibration model built using the data from the complete calibration set obtained from the master instrument is then applied to the data of the subset of samples obtained on the slave instruments. Optimal multiplicative and offset adjustments for each instrument are then calculated using linear regression of the predicted y values obtained from the slave instrument spectra versus the known y values. 

Figure 9.12 Plot of log Ki0Q versus log Kiow for a alkylated and ha-logenated (R, = alkyl, halogen) phenylureas (R2 = R3 = H A, halogen, see margin below), phe-nyl-methylureas (R2 = CH3, R3 = H, d), and phenyl-dimethylureas (R2 = R3 = CH3, ). The slope and intercept of the linear regression using all the data is given in Table 9.2 (Eq. 9-26i) each subset of ureas would yield a tighter correlation if considered alone (e.g., Eq. 9-26j).

---