Full Cross-Validation

The model was claimed to compute 5000-6000 molecules per min. The predictive ability of the model was validated by four approaches. In the first approach, a set of 20 compounds was randomly selected as an initial validation test set. A model was developed from the remaining 86 compounds with an MAE of 0.33, from which the test set values were then predicted. The results of this test prediction were very good and provided momentum for support of the three structure descriptors. In the second approach, a full cross-validation test of the model was investigated. The data set of 102 compounds was divided... 

The experimental crystallization temperature prediction model was evaluated using 13-segment cross-validation (full cross-validation) this can be considered acceptable for such small sample data sets, albeit only furnishing indicative estimates [2]. 

The model was also validated by full cross-validation. Four PCs were necessary to explain the most variation in the spectra (99.9%), which best described the molecular weight. The root mean square error of prediction... 

Full cross-validation was applied to all the regression models. Cross-validation is a strategy for validating calibration models based on systematically removing groups of samples in the modeling, and testing the removed... 

Savitzky Golay method, using the first derivative (left and right point = 1 polynomial order = 1) of FTIR selected spectra. The spectra used were from three regions pins 250 - 400 pins 445 - 589 pins 631 - 770 a total of 436 X-variables. The PLS analysis used every fourth wavelength from pins 250 to 490 (total of 48 channels) with mean centering, full-cross validation with 7 PCs, an outlier warning limit of 4.0 and the bilinear PLSR model. 

Validation without an independent test set. Each application of the adaptive wavelet algorithm has been applied to a training set and validated using an independent test set. If there are too few observations to allow for an independent testing and training data set, then cross validation could be used to assess the prediction performance of the statistical method. Should this be the situation, it is necessary to mention that it would be an extremely computational exercise to implement a full cross-validation routine for the AWA. That is. it would be too time consuming to leave out one observation, build the AWA model, predict the deleted observation, and then repeat this leave-one-out procedure separately. In the absence of an independent test set, a more realistic approach would be to perform cross-validation using the wavelet produced at termination of the AWA, but it is important to mention that this would not be a full validation. 

Multiscale DPLS. Before using the VS-DPLS method on the Euhact data set, it is instructive to first use the simple multiresolution approach. The data set was analysed with DPLS for different scale reconstructions, (j = 0...9 ). At each reconstruction level, a full cross-validated DPLS run estimates the optimal number of factors and calculates a regression model. The regression model is subsequently applied to the unseen validation set. Fig. 25 shows the results from the calibration using cross-validation. We see that the calibration error goes to zero after 5 PLS factors. 

Fu// cross-validation. Ideally, what is needed is a hybrid of self-prediction and training and test set validation. We need to use all observations in both model formation and validation without encountering the problems of selfprediction. Full cross-validation (Geisser, 1975 Stone, 1974) attempts to do just this. The term cross-validation is often applied to the partitioning form of training and test set validation and it is from this that full cross validation was developed (from here on we will use the term cross-validation to refer to full cross-validation). 

This method is an attempt to compromise between a full cross-validation (which is very slow but gives the best estimate of the model s performance when it is appUed to unknown samples) and a self-prediction (which is very fast but gives limited information about the predictive ability of the model). 

Being aware of the intercorrelations between fat, moisture and protein, it is interesting to compare NIR models with models based only on the other constituents. Prediction of the fat content from the moisture content in the the Isaksson et al. (23) data, using full cross-validation, gave RMSEP = 0.96 and R = 0.977. Prediction of... 

TABLE 7.3.17. Full Cross-Validation Prediction Error (RMSEP in Weight-%) Results ... 

Selecting an optimum group of descriptors is both an important and time-consuming phase in developing a predictive QSAR model. Frohlich, Wegner, and Zell introduced the incremental regularized risk minimization procedure for SVM classification and regression models, and they compared it with recursive feature elimination and with the mutual information procedure. Their first experiment considered 164 compounds that had been tested for their human intestinal absorption, whereas the second experiment modeled the aqueous solubility prediction for 1297 compounds. Structural descriptors were computed by those authors with JOELib and MOE, and full cross-validation was performed to compare the descriptor selection methods. The incremental... 

---