Predictive Error Sum of Squares

Several criteria can be used to select the best models, such as the F-test on regression, the adjusted correlation coefficient (R ad) and the PRESS [20] (Predictive error sum of squares). In general, even only adequate models show significant F values for regression, which means that the hypothesis that the independent variables have no influence on the dependent variables may not be accepted. The F value is less practical for further selection of the best model terms since it hardly makes any distinction between different predictive models. 

Figure 6.28) and the PC-model is calculated for the reduced data-set. Because the PC model of X is the product of t and p, the model predicts the held-out elements (the element xik is predicted as ttpk). Hence, by comparing the prediction of the held-out elements with their actual values, an estimate of the predictive power of the model is obtained. The usual estimator of the predictive power in PCA and PLS is prediction error sum of squares (PRESS), defined as ... 

Predictive Squared Error, PSE. The average value of the predictive error sum of squares ... 

Pratt measure —> statistical indices ( concentration indices) precision —> classification parameters prediction error sum of squares —> regression parameters predictive residual sum of squares —> regression parameters predictive square error regression parameters predictor variables = independent variables —> data set prime ID number ID numbers... 

Prediction error sum of squares (PRESS). The sum of the squared differences between the observed and estimated response by validation techniques [Allen, 1971, 1974] ... 

For each reduced data set, the model is calculated and responses for the deleted objects are predicted from the model. The squared differences between the true response and the predicted response for each object left out are added to PRESS ( prediction error sum of squares). From the final PRESS, the (or R ) and RMSEP ( root mean square error in prediction) values are usually calculated [Cruciani, Baroni et al, 1992]. 

By this validation technique, the original size of the data set (n) is preserved for the training set, by the selection of n objects with repetition in this way, the training set contains some repeated objects and the evaluation set is constituted by the objects left out [Efron, 1982,1987 Wehrens, Putter et al., 2000]. The model is calculated on the training set and responses are predicted on the evaluation set. All the squared differences between the true response and the predicted response of the objects of the evaluation set are collected in PRESS ( prediction error sum of squares). This procedure of building training sets and evaluation sets is repeated thousands of times, PRESS are summed up and the average predictive power is calculated. 

Another conceptually different approach is cross-validation. In Equation (2.19), X is regarded as a model for X, and as such the model should be able to predict the values of X. This can be checked by performing a cross-validation scheme in which parts of X are left out of the calculations and kept apart, the model is built and used to predict the left out entries. The sum of squared differences between the predicted and the real entries serves as a measure of discrepancy. All data in X are left out once, and the squared differences are summed in a so called PRESS statistics (PRediction Error Sum of Squares). The model that gives the lowest PRESS is selected and the pseudo-rank of X is defined as the number of components in that model. 

However, using the test set mean can significantly underestimate the predictive ability of a model. Ambiguity occurs with the statistic when the test set data is not evenly distributed over the range of the training set. As the variance of the external test set approaches the RMSE of the fitted model, the measure would approach zero, even though it would appear that the predictions are in accordance with the model. Consonni defined a new statistic that expresses the mean predicted error sum of squared deviations between the observed and predicted values for the test set, over the mean training set sum of squared deviations from the mean value ... 

PRESS is predicted error sum of squares deviations between the observed and measured y values... 

Cross-validation is one method to check the soundness of a statistical model (Cramer, Bunce and Patterson, 1988 Eriksson, Verhaar and Hermens, 1994). The data set is divided into groups, usually five to seven, and the model is recalculated without the data from each of the groups. Consecutively, predictions are obtained for the omitted compounds and compared to the actual data. The divergences are quantified by the prediction error sum of squares (PRESS sum of squares of predicted minus observed values), which can be transformed to a dimensionless term (Q ) by relating it to the initial sum of squares of the dependent variable (X(AT) )-... 

A more precise method that requires more computational time is cross-validation [155, 332]. It is implemented by excluding part of the data, performing PCA on the remaining data, and computing the prediction error sum of squares (PRESS) using the data retained (excluded from model development). The process is repeated until every observation is left out once. The order a is selected as that minimizes the overall PRESS. Two additional criteria for choosing the optimal number of PCs have also been proposed by Wold [332] and Krzanowski [155], related to cross-validation. Wold [332] proposed checking the ratio. 

PLSR method not only can avoid the collinearily problem, but also can filter off a part of noise by using predicted error sum of square (PRESS) computation. But it should be emphasized that PLSR is not very effective to solve nonlinear problems, since the addition of nonlinear terms can be only tried by trial and error method. 

For both SP and GSC-TARMA models, the selection of the value of <7v may not be straightforward, since a very low value may over-smooth the estimated parameter trajectories, while a high value may lead to noisy trajectories. The selection of (j1 may be guided by comparing the innovations (prediction residuals) with the parameter innovations (prediction error of the parameters) -that is, the residual sum of squares (RSS) to the parameter prediction error sum of squares (PESS) ... 

---