Leave-one-out technique

The optimal number of components from the prediction point of view can be determined by cross-validation (10). This method compares the predictive power of several models and chooses the optimal one. In our case, the models differ in the number of components. The predictive power is calculated by a leave-one-out technique, so that each sample gets predicted once from a model in the calculation of which it did not participate. This technique can also be used to determine the number of underlying factors in the predictor matrix, although if the factors are highly correlated, their number will be underestimated. In contrast to the least squares solution, PLS can estimate the regression coefficients also for underdetermined systems. In this case, it introduces some bias in trade for the (infinite) variance of the least squares solution. 

LOO technique leave-one-out technique - validation techniques (O cross-validation) loops - graph... 

The simplest and most general cross-validation procedure is the leave-one-out technique (LOO technique), where each object is taken away, one at a time. In this case, given n objects, n reduced models have to be calculated. This technique is particularly important as this deletion scheme is unique, and the predictive ability of the different models can be compared accurately. However, in several cases, the predictive ability obtained is too optimistic, particularly when the number of objects is quite large. This is because of a too small perturbation of the data when only one object is left out. 

The cancellation groups G range from 2 to m (in this case, leave-more-out coincides with the leave-one-out technique). For example, given 60 objects (n = 60), for 2, 3, 5, 10 cancellation groups G, at each time nIG objects are left in the evaluation sets, i.e. 30, 20,12, and 10 objects, respectively. 

With an insufficient number of training data, the leave-one-out technique was applied. In this procedure all available data are used for the training of the network, except the one for which a prediction or classification has to be performed. This method can be applied iteratively for each object in the data set. 

Figure 6.33 shows a 2D view of the descriptor used in training and prediction process of a Kohonen network. Because only 44 compounds were available, the leave-one-out technique was applied, where each compound was predicted by using the remaining 43 compounds in the training set. Figure 6.34 shows the correlation between experimental and predicted effective concentrations. 

FIGURE 6.34 Correlation between calculated and predicted effective concentrations (ECjq) for 44 progestagen derivatives with the leave-one-out technique. The standard deviation of the error of prediction is 0.1 uM/L. 

LOO technique = leave-one-out technique validation techniques (0 cross-validation) loops —> graph... 

With this classification scheme a PLS-DA model was built. The best fitting model included five latent variables. Figure 8.3 shows the plot of the first two latent variables of the PLS-DA model. In the plot, about 92% of the total variance of the data is represented. As it can be seen, a clear separation between the data related to patients with lung cancer and the other samples is observed. On the other hand, the samples related to postsurgery and healthy reference show some overlap. A numerical evaluation of the classification properties can be obtained by considering the cross-validation of the PLS-DA method according to a leave-one-out technique [44]. 

The predictive ability of the MLC QRAR model is compared in Fig. 9.12 with a three-variable QSAR model. The predicted value of log 1/C was calculated from the fitted log 1/C vs. k and log 1/C vs. (log Pow, acid dissociation constant pK and resonance parameter R) plots, by using the leave-one-out technique (the compound predicted was left out in the derivation of the model). As observed, the QSAR model had difficulties in predicting the toxicity of highly lipophilic phenols, as indicated by the curvature in this region. The results show that a single MLC retention... 

N mi s is the number mi sc 1 ass ified, and the figure in parentheses after the value of N mis is the number misclassified by two grades and Rg is the Spearman rank correlation coefficient for recognition by use of "the leave-one-out technique. 

When applied to QSAR studies, the activity of molecule u is calculated simply as the average activity of the K nearest neighbors of molecule u. An optimal K value is selected by the optimization through the classification of a test set of samples or by the leave-one-out cross-validation. Many variations of the kNN method have been proposed in the past, and new and fast algorithms have continued to appear in recent years. The automated variable selection kNN QSAR technique optimizes the selection of descriptors to obtain the best models [20]. 

The software requires the following information the concentration and spectral data, the preprocessing selections, the maximum number of factors to estimate, and the validation approach used to choose the optimal number of factors. The maximum rank selected is 10 for constructing the model to predict the caustic concentration. The validation technique is leave-one-out cross-validation where an entire design point is left out. Tliat is, there are 12 cross validation steps and all spectra for each standard (at various temperatures) are left out of the model building phase at each step. 

Quantitative structure-activity/pharmacokinetic relationships (QSAR/ QSPKR) for a series of synthesized DHPs and pyridines as Pgp (type I (100) II (101)) inhibitors was generated by 3D molecular modelling using SYBYL and KowWin programs. A multivariate statistical technique, partial least square (PLS) regression, was applied to derive a QSAR model for Pgp inhibition and QSPKR models. Cross-validation using the leave-one-out method was performed to evaluate the predictive performance of models. For Pgp reversal, the model obtained by PLS could account for most of the variation in Pgp inhibition (R2 = 0.76) with fair predictive performance (Q2 = 0.62). Nine structurally related 1,4-DHPs drugs were used for QSPKR analysis. The models could explain the majority of the variation in clearance (R2 = 0.90), and cross-validation confirmed the prediction ability (Q2 = 0.69) [ 129]. 

When the number of objects is not too small, more realistic predictive abilities are obtained by deleting more than one object at each step. To apply this cross-validation procedure, called the leave-more-out technique (LMO technique), the number of cancellation groups is defined by the user, i.e. the number of blocks the data are divided into, and, at each step, all the objects belonging to a block are left out from the calculation of the model. 

Information about the validation of the model. This section describes how the model has been validated and reports the statistics obtained by techniques such as leave-one-out cross validation, leave-many-out cross validation, Y-scrambling, and external validation. The chemicals forming the test set are also listed in this section. 

---