Backward elimination, stepwise regression

Table 4.1. models (using backwards elimination stepwise multiple regression) for the study lakes based on suites of parameters. The global indices based on annual and seasonal data, with and without lags (1 and 2-year) and deposition provided the best models. Local and regional-scale climate parameters (e.g., precipitation, temperature, etc.) did not provide significant models... 

Backwards elimination is similar to forward selection, except that the initial model contains all the covariates and removal from the model starts with the covariate of the least significance. Removal from the model then proceeds one variable at a time until no covariates meet the criteria for removal (Fout). Stepwise regression is a blend of both forward and backwards selection in that variables can be added or removed from the model at each stage. Thus, a variable may be added and a variable may be removed in the same step. 

Backward elimination starts from all variables and eliminates the one that contributes least to the model (i.e., which produces the lowest partial Ftest value). This procedure is repeated until no more X variables can be eliminated because the remaining ones are all justified by their sequential F values. Flowever, also this procedure often ends up in a local optimum in addition, it cannot be applied if the number of tested variables is larger than the number of objects. Stepwise regression avoids some of these problems. It is a forward selection procedure where, after every addition of a new variable, the possible elimination of any other one is checked by a sequential F test [41]. 

Stepwise regression proposed by Efroymson, is a combination of forward inclusion and backward elimination. After each variable is added (other than the first two), a test is performed to see if any of the variables entered at an earlier step can be deleted. The procedure applies both Eqs. [32] and [33] in a sequential manner. The stepping stops when no more variables satisfy either the criterion for removal or the criterion for inclusion. To prevent the procedure from unnecessarily cycling the critical values of P-to-enter and P-to-remove should be such that Premove < Center-... 

Linear models were generated using multiple linear regression analysis techniques. Several methods were utilized to develop models for evaluation, including stepwise addition, backward elimination, and leaps and bounds regression techniques. The models were evaluated with respect to the multiple correlation coefficient (r), the standard error (s), and predictive ability of the model. 

Several model-building techniques were elaborated forward selection and backward elimination, both in stepwise manner, all possible regressions, etc. [8]. 

---