Predicted residual sums of squares PRESS

In any case, the cross-validation process is repeated a number of times and the squared prediction errors are summed. This leads to a statistic [predicted residual sum of squares (PRESS), the sum of the squared errors] that varies as a function of model dimensionality. Typically a graph (PRESS plot) is used to draw conclusions. The best number of components is the one that minimises the overall prediction error (see Figure 4.16). Sometimes it is possible (depending on the software you can handle) to visualise in detail how the samples behaved in the LOOCV process and, thus, detect if some sample can be considered an outlier (see Figure 4.16a). Although Figure 4.16b is close to an ideal situation because the first minimum is very well defined, two different situations frequently occur ... 

For PLS solution basis sets, bulk spectra were generated as described above. Standard error of calibration values (SECV) were determined from prediction residual sum of squares (PRESS) analyses of various permutations of the amide I, II, and III bands (always including amide I) from both Ge and ZnSe spectra. After determination of the effects of different types of normalization on the results, these bands were individually normalized to an area of 100 absorbance units before PLS 1 training. 

Predictive Residual Sum of Squares, PRESS. The sum of squared differences between the observed and estimated response by validation techniques ... 

A commonly used measure of quality for a P-matrix analysis is the predicted residual sum of squares (PRESS) value computed by... 

Root mean square error in prediction (RMSEP) (or root mean square deviation in prediction, RMSDP). Also known as standard error in prediction (SEP) or standard deviation error in prediction SDEP), is a function of the prediction residual sum of squares PRESS, defined as... 

Error types can be e.g. root mean square error of cross validation (RMSECV), root mean square error of prediction (RMSEP) or predictive residual sum of squares (PRESS). 

In this section, we will describe three regression criteria relevant to Section 3.5. These criteria can be used to assess how well a model is performing. The three criteria are - the residual sum of squares (RSS), the R-squared (R ) measure and the predictive residual sum of squares (PRESS). The residual sum of squares and R-squared criteria both measure how well the model fits the data. These criteria are respectively defined... 

Analogous to the correlation coefficient in eq. (17.10), we want a measure of the quality of fit produced by a given correlation model. Two commonly used quantities are the Predicted REsidual Sum of Squares (PRESS) and the correlation coefficient R defined by the normalized PRESS value and the variance of the y data (c/). 

After developing a model, the deleted data are used as a test set, and the differences between actual and predicted Y values are calculated for the test set. The sum of squares (SS) of these differences is computed and collected from all the parallel models to form the predictive residual sum of squares (PRESS), which is a measure of the predictive ability of the model. 

Another measure for the precision of multivariate calibration is the so-called PRESS-value (predictive residual sum of squares, see Frank and Todeschini [1994]), defined as... 

This work was supported by a grant from the National Science Foundation, t Abbreviations used are as follows. FTIR Fourier transform infrared spectroscopy, ATR attenuated total reflectance, IRE internal reflection element, SATR solution ATR FTIR, FSD Fourier self-deconvolution, PLS partial least-squares analysis, PRESS prediction residual sum of squares from PLS. SECV standard error of calibration values from PLS, PLSl PLS analysis in which each component is predicted independently, PLS2 PLS analysis in which all components are predicted simultaneously. 

For each reduced data set, the model is calculated, and responses for the deleted objects are predicted from the model. The squared differences between the true response and the predicted response for each object left out are added to PRESS (predictive residual sum of squares). From the final PRESS, the (or R cv) and SDEP (standard deviation error of prediction) values are usually calculated [Cruciani et ah, 1992]. 

The next step, is to leave out part of the data from the residual matrix and compute the next component from the truncated data matrix. The left-out data can then be predicted from the expanded model and the error of prediction fir - ir ir s determined. This is repeated over and over again, until all elements in E have been left out once and only once. The prediction enor sum of squares, PRESS = 22yij, is computed from the estimated errors of prediction. If it should be found that PRESS exceeds the residual sum of squares RSS calculated firom the smaller model, the new component does not improve the prediction and is considered to be insignificant. A ratio PRESS/RSS > 1 implies that the new component predicts more noise than it explains the variation. 

To assess the predictive ability of a QSAR in the frame of MTD method the cross-validation technique is used, in which one supposes that one or more of the known experimental values are in fact unknown . The analysis is repeated, excluding the temporarily unknown compotmds. The resulting equations are used to predict the experimental measurements for the omitted compound(s), and the resulting individual squared errors of prediction are accumulated. The cross-validation cycle is repeated, leaving one out (LOO) or more (LMO) different compotmd(s), until each compound has been excluded and predicted exactly once. The result of cross-validation is the predictive discrepancy sum of squares, sometimes called PRESS (Predictive REsidual Sum of Squares) ... 

Note that this sum of squares looks similar to the residual sum of squares (RSS) given by eqn (6.12) but is different in eqn (6.12) the j/i is predicted from an equation that includes that data point here the y, is not in the model hence the term predictive residual sum of squares. The difference in predictive ability of two PLS models can be evaluated by comparison of their PRESS values. 

The OUTPUT stat ent specifies that each prediction residual (PRESS = PRES) be listed in the output. The PROC REG specifies that the following prediction residual sum of squares statistic also be generated. 

An alternative error measure is the PRESS (predicted residual sums of squares) ... 

Fortunately, since we also have concentration values for our samples, We have another way of deciding how many factors to keep. We can create calibrations with different numbers of basis vectors and evaluate which of these calibrations provides the best predictions of the concentrations in independent unknown samples. Recall that we do this by examing the Predicted Residual Error Sum-of Squares (PRESS) for the predicted concentrations of validation samples. 

Just as we did for PCR, we must determine the optimum number of PLS factors (rank) to use for this calibration. Since we have validation samples which were held in reserve, we can examine the Predicted Residual Error Sum of Squares (PRESS) for an independent validation set as a function of the number of PLS factors used for the prediction. Figure 54 contains plots of the PRESS values we get when we use the calibrations generated with training sets A1 and A2 to predict the concentrations in the validation set A3. We plot PRESS as a function of the rank (number of factors) used for the calibration. Using our system of nomenclature, the PRESS values obtained by using the calibrations from A1 to predict A3 are named PLSPRESS13. The PRESS values obtained by using the calibrations from A2 to predict the concentrations in A3... 

The Predicted Residual Error Sum of Squares (PRESS) is simply the sum of the squares of all the errors of all of the samples in a sample set. 

MSE is preferably used during the development and optimization of models but is less useful for practical applications because it has not the units of the predicted property. A similar widely used measure is predicted residual error sum of squares (PRESS), the sum of the squared errors it is often applied in CV. 

---