Validation procedure, regression objectives

Cross-validation, in which objects are eliminated and only the excluded objects are predicted from the resulting model to check its stability and validity (see chapter 5.3 for a detailed description), seems to be a too crude instrument to (automatically) decide on the validity of a QSAR regression equation. Cross-validation may be applied to relatively large data sets. But if only few compounds are included in the QSAR equation, if a certain parameter is mainly based on a single data point, or if the compounds have been selected according to a rational design procedure, e.g. a D-optimal design (chapter 6), cross-validation may incorrectly indicate a lack of validity of the QSAR model. 

If data of the real system is available, the developed simulation model can be tested for similarity with the real system in a qnantitative way (bottom-right cell in Table 4.8). For this purpose, a lot of statistical procedures can be applied depending on the specific object to be tested. Typically, regression techniqnes, distribution tests, or time series analysis methods are used. A reliable qnantitative approach is to generate a forecast of the near future by means of the simulation model which is then compared with the real systems behaviour after the forecast period has expired. This is called predictive validation A mixture of trace analysis and fixed-value test is the trace-driven simulation where a historical situation is simulated. The model s output is compared with the historical records then. 

---