Predictions, SDEP

The optimal complexity of the PLS model, that is, the most appropriate number of latent variables, is determined by evaluating, with a proper validation strategy (see Section Vl.F), the prediction error corresponding to models with increasing complexity. The parameter considered is usually the standard deviation of the error of calibration (SDEC), if computed with the objects used for building the model, or the standard deviation of the error of prediction (SDEP), if computed with objects not used for building the model (see Section Vl.F). 

Standard Deviation Error of Prediction, SDEP (. standard error in prediction, SEP). A function of the predictive residual sum of squares, defined as ... 

Depczynski fitness function. A parameter based on the standard deviation error both in calculation SDEC and prediction SDEP ]Depczynsld, Erost et al, 2000] ... 

These measures can, of course, be equivalently expressed as residual standard deviations (RSDs) and residual predictive standard deviations (PRESDs). The latter is often called the standard error of prediction (SDEP). If one has some knowledge of the noise in the investigated system, for example 0.3 unit for log (1/C), these standard deviations should, of course, be similar in size. 

One of the first studies to predict log P by using potential energy fields calculated using the GRID and CoMFA approaches was done by Kim [60]. The author investigated H, CH3 and H2O probes, and calculated the best models using the hydro-phobic probe H2O for relatively small series (20 or less compounds each) of furans, carbamates, pyridines and pyrazines. A similar study was performed by Waller [61] who predicted a small series of 24 polyhalogenated compounds. Recently, Caron and Ermondi [62] used a new version of Cruciani s software, VolSurf [63], to predict the octanol-water and alkane-water partition coefficients for 152 compounds with r = 0.77, q = 0.72, SDEP = 0.60 for octanol-water and r = 0.76, q = 0.71, SDEP = 0.85 for alkane-water. 

The LOO cross-validated q oo values for the initial models was 0.875 using the water probe and 0.850 using the methyl probe. The application of the SRD/FFD variable selection resulted in an improvement of the significance of both models. The analysis yielded a correlation coefficient with a cross-validated q Loo of 0.937 for the water probe and 0.923 for the methyl probe. In addition we tested the reliability of the models by applying leave-20%-out and leave-50%-out cross-validation. Both models are also robust, indicated by high correlation coefficients of = 0.910 (water probe, SDEP = 0.409) and 0.895 (methyl probe, SDEP = 0.440) obtained by using the leave-50%-out cross-validation procedure. The statistical results gave confidence that the derived model could also be used for the prediction of novel compounds. 

On the basis of the above considerations we used a test set of 105 compounds for external validation of the VolSurf model. Test set compounds are listed together with their experimental and calculated aqueous solubility values in Table 8.2. Projection of the test set predictions into the VolSurf training set model is documented in Fig. 8.3 the SDEP (standard deviation of the error of prediction) value amounts to 0.99. The black dots nicely prove that the majority of the 105 test set structures were well predicted. 

It is extremely difficult to find compounds with experimental VD equivalent to those collected by Lombardo. We could detect only 10 compounds, which were in turn used as test set for external validation of the VolSurf library model. Test set compounds are listed together with their experimental and calculated VD values in Table 8.4 the projection of their predictions is plotted in Figure 8.5 (filled dots) the SDEP value amounts to 0.53. 

G. Costantino, D. Riganelli, B. Skager-berg. Predictive Ability of Regression Models. Part I Standard Deviation of Prediction Errors (SDEP),/. Chemomet. 1992, 6, 335-346. 

For each reduced data set, the model is calculated, and responses for the deleted objects are predicted from the model. The squared differences between the true response and the predicted response for each object left out are added to PRESS (predictive residual sum of squares). From the final PRESS, the (or R cv) and SDEP (standard deviation error of prediction) values are usually calculated [Cruciani et ah, 1992]. 

RMSEP = RMSDP = SEP = SDEP = where PSE is the predictive square error. 

Afzelius et al. [112] applied GRIND descriptors to generate quantitative and qualitative models for CYP2C9 inhibition. The resulting PLS model gave loo (Jlgo values of 0.60 and 0.57, respectively. Application to an external test set of 12 compounds gave a standard deviation of prediction errors (SDEP) of 0.37 log units ... 

Figure 3. Statistics derived from a comparative molecular similarity indices analysis (CoMSIA) [26] for the agrophore model shown in Figure 2A. The graph depicts the predictive power of a leave-one-out cross-validation procedure for 318 Protox inhibitors (SDEP = standard errorof prediction).

A parameter that does not depend on the mean target property, the standard deviation of error of prediction, or SDEP, has been proposed 21... 

Figure 4 Predictive power analysis actual versus predicted pK, (p,M) activities for 17 aromatase inhibitors, based on the 33-compound model with the lowest PRESS and SDEP values (the two-component model). Data modified from Ref. 96.

---