Best subset regression

To carry out an all subset regression means that all models will be constructed of size one (of which there are 30), of size two (there are 

Model size 2 4 Model size 3 4 Model size 4 4 

Model size 5 Model size 6 Model size 7  

---

The following three performance measures are commonly used for variable selection by stepwise regression or by best-subset regression. An example in Section 4.5.8 describes use and comparison of these measures. 

Best-subset Best-subset regression select the solution with the smallest BIC value. 

Hofmann, M., Gatu, C., Kontoghiorghes, E. J. Computat. Stat. Data Anal. 52, 2007, 16-29. Efficient algorithms for computing the best subset regression models for large-scale problems. 

In many cases, it is possible to use only a subset of the p variables X without a serious loss predictive ability with the forward stepwise regression (SWMLR) procedure, which, in each step, selects the predictor that more increases the variation explained and verifies if a previously selected predictor can be removed (values for F-statistics to enter and to remove variables should be fixed) or with the best subsets regression procedure. 

Salt, D.W., Ajmani, S., Chrichton, R. and Livingstone, D.J. (2007a) An improved approximation to the estimation of the critical F values in best subset regression. J. Chem. Inf. Model., 47, 143—149. 

Table 7 Best Subset Regression Models of Size 2 to 9... 

A correct LOO cross-validation can be done by moving the delete-and-predict step inside the subset search loop. In other words, we take the sample of size n, remove one case, search for the best subset regression on the remaining n - cases, and apply this subset regression to predict the holdout case. Repeat for each of the cases in turn, gethng a true holdout prediction for each of the cases. Use these holdouts as a measure of the fit [38,39]. 

---