Leave-one-out procedure

The value in brackets are the 95% confidence intervals, and Q,is the correlation coefficient of the plot y0ts vs. ypred by the leave-one-out procedure. 

Table 4.2 shows the F values predicted after the leave one out procedure using Eq. (12). For all the compounds assayed, the prediction is fairly good. The predicted F values correlate very well with those determined in vivo from plasma level data. 

H K -ATPase inhibitory activity Predicted activity by leave one out procedure... 

Validate this HDP by a standard cross-validation procedure, which generates the cross-validated (or q ) value for the kNN-QSAR model built by use of this HDP. The standard leave-one-out procedure has been implemented as follows (0 Eliminate a compound from the training set. Hi) Calculate the activity of the eliminated compound, which is treated as an unknown, as the average activity of the k most similar compounds found in the remaining molecules (k is set to 1 initially). The similarities between compounds are calculated using only the selected descriptors (i.e., the current trial HDP) instead of the whole set of descriptors, iiii) Repeat this procedure until every compound in the training set has been eliminated and predicted once, iiv)... 

These relationships highlight how the leave-one-out procedure may give an over-optimistic prediction ability when the number of samples is high enough. Then, adjusted R can be viewed as an estimate of Q loo for an infinite number of samples ... 

Validation without an independent test set. Each application of the adaptive wavelet algorithm has been applied to a training set and validated using an independent test set. If there are too few observations to allow for an independent testing and training data set, then cross validation could be used to assess the prediction performance of the statistical method. Should this be the situation, it is necessary to mention that it would be an extremely computational exercise to implement a full cross-validation routine for the AWA. That is. it would be too time consuming to leave out one observation, build the AWA model, predict the deleted observation, and then repeat this leave-one-out procedure separately. In the absence of an independent test set, a more realistic approach would be to perform cross-validation using the wavelet produced at termination of the AWA, but it is important to mention that this would not be a full validation. 

This means that Wsoried can be divided into two submatrices, Ws and W , containing important and noisy coefficients, respectively (see Scheme 2b). Then, using a leave-one-out procedure we can construct the PLS model with A factors and calculate the matrix of b-coefficients, their means and standard deviations, and finally their stability (Scheme 3). The stability of the regression coefficients associated with the noisy features can then be used to calculate a threshold value, which allows to distinguish relevant and irrelevant features within the group of n original features. 

Abe et. at. C23 reported another study on the verification of correlations between mass spectra and biological activity. Several pattern recognition methods have been applied to a set of 17 analgesics and 16 antispasmodics. Predictive abilities of more than 90 % have been obtained by the KNN-method and by the learning machine. A set of 30 features and the leave-one-out-procedure was employed. 

---