Squared prediction error statistic

Several statistics from the models can be used to monitor the performance of the controller. Square prediction error (SPE) gives an indication of the quality of the PLS model. If the correlation of all variables remains the same, the SPE value should be low, and indicate that the model is operating within the limits for which it was developed. Hotelling s 7 provides an indication of where the process is operating relative to the conditions used to develop the PLS model, while the Q statistic is a measure of the variability of a sample s response relative to the model. Thus the use of a multivariate model (PCA or PLS) within a control system can provide information on the status of the control system. 

In any case, the cross-validation process is repeated a number of times and the squared prediction errors are summed. This leads to a statistic [predicted residual sum of squares (PRESS), the sum of the squared errors] that varies as a function of model dimensionality. Typically a graph (PRESS plot) is used to draw conclusions. The best number of components is the one that minimises the overall prediction error (see Figure 4.16). Sometimes it is possible (depending on the software you can handle) to visualise in detail how the samples behaved in the LOOCV process and, thus, detect if some sample can be considered an outlier (see Figure 4.16a). Although Figure 4.16b is close to an ideal situation because the first minimum is very well defined, two different situations frequently occur ... 

The goal of the RMSECV, statistic is twofold. It yields an estimate of the root mean squared prediction error E(y-y)2 when k components are used in the model, whereas the curve of RMSECV, for k = 1,. .., k,ma is a popular graphical tool to choose the optimal number of components. 

Multivariate monitoring charts based on Hotelling s statistic (T ) and squared prediction errors SPEx and SPEy) are constructed using the PLS models. Hotelling s statistic for a new independent t vector is [298]... 

As shown above, the residuals will not be independent of the data in the case of component modeling when individual samples are left out during the cross-validation. In regression though, the y and yco are statistically independent and Equation (7.3) is an estimator of the summed squared prediction error of y using a certain model. 

In this paper, the building of a statistical model to diagnose the fault condition of gearboxes based on a framework of Time Domain Averaging across all Scales - IDAS, proposed by HALIM et al. (2008), is undertaken. This model pays special attention to statistical analysis using the concept of squared prediction errors, which allows the most likely condition of gearboxes to be estimated. 

In any case, the crossvalidation process is repeated a number of times and the squared prediction errors are summed. This leads to a statistic [the predicted... 

The g-statistic or square of predicted errors (SPE) is the sum of squares of the errors between the data and the estimates, a direct calculation of variability ... 

All regression methods aim at the minimization of residuals, for instance minimization of the sum of the squared residuals. It is essential to focus on minimal prediction errors for new cases—the test set—but not (only) for the calibration set from which the model has been created. It is relatively easy to create a model— especially with many variables and eventually nonlinear features—that very well fits the calibration data however, it may be useless for new cases. This effect of overfitting is a crucial topic in model creation. Definition of appropriate criteria for the performance of regression models is not trivial. About a dozen different criteria— sometimes under different names—are used in chemometrics, and some others are waiting in the statistical literature for being detected by chemometricians a basic treatment of the criteria and the methods how to estimate them is given in Section 4.2. 

Root Mem Square Error of Prediction (RMSEP) (Model Diagnostic) The RMSEP is anciaer diagnostic for examining the errors in the predicted concentrations. Whie the statistical prediction error discussed earlier quantifies preci-... 

ANOVA of the data confirms that there is a statistically significant relationship between the variables at the 99% confidence level. The i -squared statistic indicates that the model as fitted explains 96.2% of the variability. The adjusted P-squared statistic, which is more suitable for comparing models with different numbers of independent variables, is 94.2%. The prediction error of the model is less than 10%. Results of this protocol are displayed in Table 18.1. Validation results of the regression model are displayed in Table 18.2. 

Table IV shows the overall analysis of variance (ANOVA) and lists some miscellaneous statistics. The ANOVA table breaks down the total sum of squares for the response variable into the portion attributable to the model, Equation 3, and the portion the model does not account for, which is attributed to error. The mean square for error is an estimate of the variance of the residuals — differences between observed values of suspensibility and those predicted by the empirical equation. The F-value provides a method for testing how well the model as a whole — after adjusting for the mean — accounts for the variation in suspensibility. A small value for the significance probability, labelled PR> F and 0.0006 in this case, indicates that the correlation is significant. The R2 (correlation coefficient) value of 0.90S5 indicates that Equation 3 accounts for 91% of the experimental variation in suspensibility. The coefficient of variation (C.V.) is a measure of the amount variation in suspensibility. It is equal to the standard deviation of the response variable (STD DEV) expressed as a percentage of the mean of the response response variable (SUSP MEAN). Since the coefficient of variation is unitless, it is often preferred for estimating the goodness of fit.

Models can be generated using stepwise addition multiple linear regression as the descriptor selection criterion. Leaps-and-bounds regression [10] and simulated annealing (ANNUN) can be used to find a subset of descriptors that yield a statistically sound model. The best descriptor subset found with multiple linear regression can also be used to build a computational neural network model. The root mean square (rms) errors and the predictive power of the neural network model are usually improved due to the higher number of adjustable parameters and nonlinear behavior of the computational neural network model. 

Pratt measure —> statistical indices ( concentration indices) precision —> classification parameters prediction error sum of squares —> regression parameters predictive residual sum of squares —> regression parameters predictive square error regression parameters predictor variables = independent variables —> data set prime ID number ID numbers... 

Another conceptually different approach is cross-validation. In Equation (2.19), X is regarded as a model for X, and as such the model should be able to predict the values of X. This can be checked by performing a cross-validation scheme in which parts of X are left out of the calculations and kept apart, the model is built and used to predict the left out entries. The sum of squared differences between the predicted and the real entries serves as a measure of discrepancy. All data in X are left out once, and the squared differences are summed in a so called PRESS statistics (PRediction Error Sum of Squares). The model that gives the lowest PRESS is selected and the pseudo-rank of X is defined as the number of components in that model. 

However, using the test set mean can significantly underestimate the predictive ability of a model. Ambiguity occurs with the statistic when the test set data is not evenly distributed over the range of the training set. As the variance of the external test set approaches the RMSE of the fitted model, the measure would approach zero, even though it would appear that the predictions are in accordance with the model. Consonni defined a new statistic that expresses the mean predicted error sum of squared deviations between the observed and predicted values for the test set, over the mean training set sum of squared deviations from the mean value ... 

Cross-validation is one method to check the soundness of a statistical model (Cramer, Bunce and Patterson, 1988 Eriksson, Verhaar and Hermens, 1994). The data set is divided into groups, usually five to seven, and the model is recalculated without the data from each of the groups. Consecutively, predictions are obtained for the omitted compounds and compared to the actual data. The divergences are quantified by the prediction error sum of squares (PRESS sum of squares of predicted minus observed values), which can be transformed to a dimensionless term (Q ) by relating it to the initial sum of squares of the dependent variable (X(AT) )-... 

In a multivariate calibration, where a set of NIR spectra (Xnxk, N samples and K variables) is regressed onto a continuous variable (yivxi) such as the fat or moisture content, the statistical errors, the accuracy, are most often used as a quality measure of the calibration. The absolutely most common quality measure of a multivariate calibration is the prediction error, expressed either as root mean square error of prediction (RMSEP) or standard error of performance (SEP). Both are calculated and are the result of a validation process, such as test set or cross-validation. These prediction errors are defined as ... 

---