Parametric Statistics in Linear and Multiple Regression

The problem of over-optimistic estimates of model quality is a general one for all multivariate regression methods, and a number of model quality diagnostics have been developed that do not rely on parametric assumptions, to both limit the model fitting process and to assess the ability of the model to generalise beyond the training set. 

An alternative approach to using traditional parametric statistical methods to calculate the significance of fitted correlations, would be to directly assess the model based on its ability to predict, rather than merely to assess how well the model fits the training set. When the quality of the model is assessed by the prediction of a test set, rather than the fit of the model to its training set, a statistic related to r or can be defined, and denoted q or q, to indicate that the quality measure is assessed in prediction. A q may be calculated by internal cross-validation techniques, or by the quality of predictions of an independent test set, in which case an upper-case is used. The equation to calculate q (or Q ) is shown in equation 9.3. 

Cross-validation can only give an unbiased estimate of future predictive ability if future compounds come from the same population as those in the 

---