Sum of square error

We can also examine these results numerically. One of the best ways to do this is by examining the Predicted Residual Error Sum-of-Squares or PRESS. To calculate PRESS we compute the errors between the expected and predicted values for all of the samples, square them, and sum them together. 

PRESS for validation data. One of the best ways to determine how many factors to use in a PCR calibration is to generate a calibration for every possible rank (number of factors retained) and use each calibration to predict the concentrations for a set of independently measured, independent validation samples. We calculate the predicted residual error sum-of-squares, or PRESS, for each calibration according to equation [24], and choose the calibration that provides the best results. The number of factors used in that calibration is the optimal rank for that system. 

Fortunately, since we also have concentration values for our samples, We have another way of deciding how many factors to keep. We can create calibrations with different numbers of basis vectors and evaluate which of these calibrations provides the best predictions of the concentrations in independent unknown samples. Recall that we do this by examing the Predicted Residual Error Sum-of Squares (PRESS) for the predicted concentrations of validation samples. 

Just as we did for PCR, we must determine the optimum number of PLS factors (rank) to use for this calibration. Since we have validation samples which were held in reserve, we can examine the Predicted Residual Error Sum of Squares (PRESS) for an independent validation set as a function of the number of PLS factors used for the prediction. Figure 54 contains plots of the PRESS values we get when we use the calibrations generated with training sets A1 and A2 to predict the concentrations in the validation set A3. We plot PRESS as a function of the rank (number of factors) used for the calibration. Using our system of nomenclature, the PRESS values obtained by using the calibrations from A1 to predict A3 are named PLSPRESS13. The PRESS values obtained by using the calibrations from A2 to predict the concentrations in A3... 

The Predicted Residual Error Sum of Squares (PRESS) is simply the sum of the squares of all the errors of all of the samples in a sample set. 

An F-test for lack of fit is based on the ratio of the lack of fit sum to the pure error sum of squares divided by their corresponding degrees of freedom ... 

Van der Voet [21] advocates the use of a randomization test (cf. Section 12.3) to choose among different models. Under the hypothesis of equivalent prediction performance of two models, A and B, the errors obtained with these two models come from one and the same distribution. It is then allowed to exchange the observed errors, and c,b, for the ith sample that are associated with the two models. In the randomization test this is actually done in half of the cases. For each object i the two residuals are swapped or not, each with a probability 0.5. Thus, for all objects in the calibration set about half will retain the original residuals, for the other half they are exchanged. One now computes the error sum of squares for each of the two sets of residuals, and from that the ratio F = SSE/JSSE. Repeating the process some 100-2(K) times yields a distribution of such F-ratios, which serves as a reference distribution for the actually observed F-ratio. When for instance the observed ratio lies in the extreme higher tail of the simulated distribution one may... 

The cumulative curve obtained from the transit time distribution in Figure 9 was fitted by Eq. (48) to determine the number of compartments. An additional compartment was added until the reduction in residual (error) sum of squares (SSE) with an additional compartment becomes small. An F test was not used, because the compartmental model with a fixed number of compartments contains no parameters. SSE then became the only criterion to select the best compartmental model. The number of compartments generating the smallest SSE was seven. The seven-compartment model was thereafter referred to as the compartmental transit model. 

The answer to this question is in the residuals. While the residuals might not seem to bear any relationship to either the original data or the errors (which in this case we know because we created them and they are listed above), in fact the residuals contain the variance present in the errors of the original data. However, the value of the error sum of squares is reduced from that of the original data, because of the subtraction of some fraction of the error variation from the total when the row and column means were subtracted from the data itself. This reduction in the sum of squares can be compensated for by making a corresponding compensation in the degrees of freedom used to calculate the mean square from the sum of squares. In this data the sum of squares of the residuals is 5.24 (check it out). 

If there are n replications at q different settings of the independent variables, then the pure-error sum of squares is said to possess (n — 1) degrees of freedom (1 degree of freedom being used to estimate y) while the lack-of-fit sum of squares is said to possess N — p — q(n — 1) degrees of freedom, i.e., the difference between the degrees of freedom of the residual sum of squares and the pure-error sum of squares. 

This is, then, the regression sum of squares due to the first-order terms of Eq. (69). Then, we calculate the regression sum of squares using the complete second-order model of Eq. (69). The difference between these two sums of squares is the extra regression sum of squares due to the second-order terms. The residual sum of squares is calculated as before using the second-order model of Eq. (69) the lack-of-fit and pure-error sums of squares are thus the same as in Table IV. The ratio contained in Eq. (68) still tests the adequacy of Eq. (69). Since the ratio of lack-of-fit to pure-error mean squares in Table VII is smaller than the F statistic, there is no evidence of lack of fit hence, the residual mean square can be considered to be an estimate of the experimental error variance. The ratio... 

MSE is preferably used during the development and optimization of models but is less useful for practical applications because it has not the units of the predicted property. A similar widely used measure is predicted residual error sum of squares (PRESS), the sum of the squared errors it is often applied in CV. 

PRESS Predicted residual error sum of squares (sum of squared prediction errors). rjk (Pearson) correlation coefficient between variables j and k r2 is the... 

Several criteria can be used to select the best models, such as the F-test on regression, the adjusted correlation coefficient (R ad) and the PRESS [20] (Predictive error sum of squares). In general, even only adequate models show significant F values for regression, which means that the hypothesis that the independent variables have no influence on the dependent variables may not be accepted. The F value is less practical for further selection of the best model terms since it hardly makes any distinction between different predictive models. 

Figure 6.28) and the PC-model is calculated for the reduced data-set. Because the PC model of X is the product of t and p, the model predicts the held-out elements (the element xik is predicted as ttpk). Hence, by comparing the prediction of the held-out elements with their actual values, an estimate of the predictive power of the model is obtained. The usual estimator of the predictive power in PCA and PLS is prediction error sum of squares (PRESS), defined as ... 

In this equation, the term SSW is refereed to as the the sum of squares within groups or error sum of squares. The quantity SSW when divided by the appropriate degrees of freedom J(I-l) is referred to as the mean square or error mean square and is denoted by MSW- As Eq. (1.114) is not particularly convenient for calculation purposes, it can be presented in the more usable form ... 

To determine the minimum, the partial derivative of the error sum of squares with respect to each constant is set equal to zero to yield ... 

Note that in preliminary calculations the sum of replicated design points-trials is taken as the response, and thus the number of replicated design points n is introduced Eq. (2.70). As there exists replication of trials, it is evident that the error sum of squares is calculated in accord with analysis of variance methodology. To enable comparison of such variance determination with classical analysis of variance, it is necessary to transform Table 2.107 into Table 2.108. 

Sometimes the question arises whether it is possible to find an optimum regression model by a feature selection procedure. The usual way is to select the model which gives the minimum predictive residual error sum of squares, PRESS (see Section 5.7.2) from a series of calibration sets. Commonly these series are created by so-called cross-validation procedures applied to one and the same set of calibration experiments. In the same way PRESS may be calculated for a different sets of features, which enables one to find the optimum set . 

The PLS model is calculated without these values. The omitted values are predicted and then compared with the original values. This procedure is repeated until all values have been omitted once. Therefore an error of prediction, in terms of its dependence on the number of latent variables, is determined. The predicted residual error sum of squares (PRESS) is also the parameter which limits the number of latent vectors u and t ... 

The suitability of the regression model should be proven by a special statistical lack-of-fit-test, which is based on an analysis of variance (ANOVA). Here the residual sum of squares of regression is separated into two components the sum of squares from lack-of-fit (LOF) and the pure error sum of squares (PE, pure errors)... 

These two quantities are often referred to as the error sum of squares and the error degrees of freedom, respectively. The former divided by the latter is a statistical value that is the best estimate of the variance, a2, common to all k populations ... 

---