Validation of the predicting function

It is important to assess how well the predicting function fits the observed values of the dependent variable. Useful statistics are RSS in the case of a regression and TCE for classification problems. The calculation of these quantities is a resubstitutton, since the values used to obtain the predicting function are the same as the values used to validate the function. A typical statistic for validation of predicting function / in a regression is the multiple correlation coefficient 

The disadvantage of R is that it is not useful to compare predicting functions that containing an unequal numbers of predictors, since it does not decrease with increasing number of predictors. Therefore the standard error of a regression is introduced 

The complexity of the predicting function is taken into account via d, the number of its degrees of freedom. We shall discuss this in Section 6.2 for various types of predicting functions. A good predicting function has a small S value. 

Another statistic often used for regression models is the empirical F value, which is defined as 

The predictive ability of a predicting function is as important as its fit. To quantify this, the set of observations is first partitioned randomly into a learning set LS and a test set TS, the test sample  

---