Best Subset Procedures

the coefficient of determination, and SSe, the sum of squares error term, can be used to help find the best subset ( ) of x, variables. R and SSe are denoted with a subscript k for the number of x, variables in the model. When R is large, SSe tends to be small, because the regression variability is well explained by the regressors, so random error becomes smaller. 

Another way of determining the best k number of x, variables is using Adj and MSej. The model with the highest Adj R also will be the model with the smallest MSe. This method better takes into account the number of x, variables in the model. 

This value represents the total mean square error of the n fitted values for each k. 

The goal is to determine the value subset for which the value is approximately equal ( ) to k. If the model is adequate, the value is equivalent to k, the number of x, variables. A small value indicates small variance, which will not decrease further with increased numbers of k. 

Many software programs provide outputs for these subset predictors, as given in Table 10.10, for the data from Example 10.1. 

---

Table 4 lists the 10 most probable subsets found by the Bayesian procedure. With the exception of the second subset listed (BlHq, BqHq), every term in every subset has at least one lower-order effect also in the subset. For example, in the fifth subset listed in Table 4 (FL, HL, Hq, AlHq, GlHq, BlHq, ELFi), ht active effect GlHq has parent Hq which, in turn, has parent Hl. (The notions of parents and effect heredity are stated precisely in Section 2.2.) This fifth subset contains all the effects in the best subset of size 5 listed in Table 3. The Bayesian procedure has found a subset similar to one of the best subsets but which obeys effect heredity.

The Bayesian approach is more than a tool for adjusting the results of the all subsets regression by adding appropriate effects to achieve effect heredity. Take, for example, the sixth model in Table 4 which consists of Al,Bl, AlDq, BlHl, BlHq, BqHq. The AlDq effect identified as part of this model does not appear in the best subsets of size 1-6 in Table 3. The Bayesian procedure has therefore discovered an additional possible subset of effects that describes the data. 

In many cases, it is possible to use only a subset of the p variables X without a serious loss predictive ability with the forward stepwise regression (SWMLR) procedure, which, in each step, selects the predictor that more increases the variation explained and verifies if a previously selected predictor can be removed (values for F-statistics to enter and to remove variables should be fixed) or with the best subsets regression procedure. 

The two major drawbacks of the stepwise procedures are that none of them ensure that the best subset of a given size is found and, perhaps more critical, it is not uncommon that the first variable included in FS becomes unnecessary in the presence of other variables. 

Floating search method proposed by Pudil et al. [112] is used as feature subset generation procedure, and has been proved to be one of the best subset generation algorithms for moderately large or small data sets. Backward floating search (BFS) method needs more computer time than forward floating search method, but it can treat interactive features well, so we combine it with SVM based on wrapper method and name it SVM-BFS method. 

Procedures often must be limited to the existing equipment in spite of recommendations drawn from unrestricted optimization considerations. Moreover, introducing a new product into the market quickly and ahead of competitors can sometimes be more important and profitable than realizing the lowest manufacturing costs. The best process must be selected from a subset... 

Since an exhaustive search—eventually combined with exhaustive evaluation— is practically impossible, any variable selection procedure will mostly yield subopti-mal variable subsets, with the hope that they approximate the global optimum in the best possible way. A strategy could be to apply different algorithms for variable selection and save the best candidate solutions (typically 5-20 variable subsets). With this low number of potentially interesting models, it is possible to perform a detailed evaluation (like repeated double CV) in order to find one or several variables... 

A better validation strategy, in such cases, is to use three sample subsets a training set, an optimization set, and an evaluation set. The optimization set is used to find the best modeling settings, while the actual reliability of fhe final model is esfimafed by way of a real prediction on the third subset, formed by objecfs fhaf have never influenced the model. The three-set validation procedure should always be used in ANN modeling, which presents a very high risk of overfiffing. 

Such a correlation is unnecessarily divergent. An alternative is to base data reduction on just the P-Xi data subset this is possible because the full P-Xi-yi data set includes redundant information. Assuming that the correlating equation is appropriate to the data, one merely searches for values ofthe parameters a, p, etc., that yield pressures by Eq. (4-318) that are as close as possible to the measured values. The usual procedure is to minimize the sum of squares of the residuals 8P. Known as Barker s method [Austral. J. Chem. 6 207—210 (1953)], it provides the best possible fit of the experimental pressures. When experimental yt values are not consistent with the P"-Xi data. Barkers method cannot lead to calculated yi values that closely match the experimental yt values. With experimental error usually concentrated in the yt values, the calculated yi values are likely to be more nearly correct. Because Barker s method requires only the P -Xi data subset, the measurement o( yt values is not usually worth the extra effort, and the correlating parameters a, P, etc., are usually best determined without them. Hence, many P°-Xi data subsets appear in the literature they are of course not subject to a test for consistency by the Gibbs-Duhem equation. 

As in the previous section, we are interested in linear combinations of variables, with the goal of determining that combination which best summarizes the n-dimensional distribution of data. We are seeking the Unear combination with the largest variance, with normalized coefficients applied to the variables used in the linear combinations. This axis is the so-called rst principal axis or first principal component. Once this is determined, then the search proceeds to find a second normalized linear combination that has most of the remaining variance and is uncorrelated with the first principal component. The procedure is continued, usually imtil all the principial components have been calculated. In this case, p = n and a selected subset of the principal components is then used for further analysis and for interpretation. 

---