Root mean squared error of prediction

Aptula AQ, Jeliazkova NG, Schultz TW, Cronin MTD. The better predictive model high q for the training set or low root mean square error of prediction for the test set QSAR Comb Sci 2005 24 385-96. 

Root-mean-squared error of prediction (RMSEP), 6 50-51... 

The pilot study showed good prospects for predicting crystallization point temperatures directly from the acoustic signatures of the liquid feed into the granulator with an indicated prediction error (root mean square error of prediction) RMSEP = 1.1 °C. 

Chemometrics in Process Analytical Technology (PAT) 409 The main figure of merit in test set validation is the root mean square error of prediction (RMSEP) ... 

NIR models are validated in order to ensure quality in the analytical results obtained in applying the method developed to samples independent of those used in the calibration process. Although constructing the model involves the use of validation techniques that allow some basic characteristics of the model to be established, a set of samples not employed in the calibration process is required for prediction in order to conhrm the goodness of the model. Such samples can be selected from the initial set, and should possess the same properties as those in the calibration set. The quality of the results is assessed in terms of parameters such as the relative standard error of prediction (RSEP) or the root mean square error of prediction (RMSEP). 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The validation set is employed to determine the optimum number of variables to use in the model based on prediction (RMSEP) rather than fit (RMSEO- RM-SEP as a function of the number of variables is plotted in Figure 5.7S for the prediction of the caustic concentration in the validation set, Tlie cuive levels off after three variables and the RMSEP for this model is 0.053 Tliis value is within the requirements of the application (lcr= 0.1) and is not less than the error in the reported concentrations. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The new RMSEP plot in Figure 5-100 is more well behaved than the plot shown in Figure 5-93 (with the incorrect spectrum 3). A minimum is found at 3 factors with a corresponding RMSEP that is almost two orders of magnitude smaller than the minimum in Figure 5-93- The new RMSEP plot shows fairly ideal behavior with a sharp decrease in RMSEP as factors are added and then a slight increase when more than three factors are included. 

Root mean square error of prediction OtMSEP) plot (RMSBP vs. number of factors)... 

RMSEP. see Root mean square error, of prediction... 

Root Mean Square Error of Prediction (RMSEP) (Model Diagnostic) The RMSEP for the determination of caustic is 0.06 wt.% over a range of 7.4-10.4 wt.%. This estimate of prediction ability indicates that the performance of the model is acceptable for the application. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) Prediction error is a useful metric for selecting the optimum number of factors to include in the model. This is because the models are most often used to predict the concentrations in future unknown samples. There are two approaches for generating a validation set for estimating the prediction error internal validation (i.e., cross-validation with the calibration data), or external validation (i.e., perform prediction on a separate validation set). Samples are usually at a premium, and so we most often use a cross- validation approach. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The RMSEP versus number of factors plot in Figure 5.113 shows a break at three factors and a leveling off after six factors. Tlie RMSEP value with six factors (0,04) is comparable to the estimated error in the reported concentrations (0.033), indicating the model is predicting well At this point we tentatively choose a rank six model. The rank three model shows an RMSEP of 0.07 and may well have been considered to be an adequate model, depending on how well the reference values are known. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The RMSEP plot for the MCB model is shown in Figure 5.127. Although the shape of this RMSEP plot is not ideal, it does not exhibit erratic behavior. Tlie first minimum in this plot is at four factors with a lower minimum at six factors. In Section 5.2.1.2, nonlinear behavior was suspected as the root cause of the failure of the DCLS method. Tlicreforc, it is reasonable that a PLS model re-... 

The first PC quantified the moisture content as evident from the strong contribution around 1930 nm region, which represents the combination band of the -OH stretching and -OH bending vibrations. The second PC quantified the baseline shift due to changing sample density as a result of changing compression pressures. The third PC quantified the changes in MCC structure as evident from its resemblance with NIR spectrum collected on the 100% MCC powder sample. The root mean-square errors of prediction for different sample properties are summarized in Table 14. 

The new samples constitute what is called the validation set and, of course, the more validation samples you have, the more representative the conclusions will be. The RMSEP (root mean square error of prediction) statistic measures how well the model predicts new samples. It is calculated as... 

Figure 3. Reconstructions of (A) diatom-based and (B) chrysophyte-based monomeric Al for Big Moose Lake, and diatom-based monomeric Al for (C) Deep Lake, (D) Upper Wallface Pond, and (E) Windfall Pond in the Adirondack Mountains, New York. Reconstructions are bounded by bootstrapping estimates of the root mean-squared error of prediction for each sample. Bars to the right of each reconstruction indicate historical (H) and Chaoborus-based (C) reconstructions of fishery resources. The historical fish records are not continuous, unlike the paleolimnological records. Intervals older than 1884 are dated by extrapolation. (Reproduced with permission from reference 10.

At this point, it is worth noting that the same validation methods that are used to avoid overfitting of quantitative models (Section 8.3.7) can also be used to avoid overfitting in qualitative models. The only difference is that the figure of merit in this case is not the Root Mean Squared Error of Prediction (RMSEP), but rather the percent correct classification or %CC ... 

Municipal solid waste Near infrared Oxidation induction time Partial least square Post-consumer recyclate Post-consumer waste Principal component analysis Principal component regression Root-mean-square error of prediction... 

The model was also validated by full cross-validation. Four PCs were necessary to explain the most variation in the spectra (99.9%), which best described the molecular weight. The root mean square error of prediction... 

The b vector chosen by the validation procedure can be employed prospectively to predict concentrations of the analyte of interest in independent data. Similar to the calculation of RMSECV, the root mean square error of prediction (RMSEP) for an independent data set is defined as the square root of the sum of the squares of the differences between predicted and reference concentrations. 

Figures 11 and 12 illustrate the performance of the pR2 compared with several of the currently popular criteria on a specific data set resulting from one of the drug hunting projects at Eli Lilly. This data set has IC50 values for 1289 molecules. There were 2317 descriptors (or covariates) and a multiple linear regression model was used with forward variable selection the linear model was trained on half the data (selected at random) and evaluated on the other (hold-out) half. The root mean squared error of prediction (RMSE) for the test hold-out set is minimized when the model has 21 parameters. Figure 11 shows the model size chosen by several criteria applied to the training set in a forward selection for example, the pR2 chose 22 descriptors, the Bayesian Information Criterion chose 49, Leave One Out cross-validation chose 308, the adjusted R2 chose 435, and the Akaike Information Criterion chose 512 descriptors in the model. Although the pR2 criterion selected considerably fewer descriptors than the other methods, it had the best prediction performance. Also, only pR2 and BIC had better prediction on the test data set than the null model.

The two regression models have lower prediction accuracy than the random-function model when assessed using cross-validated root mean squared error of prediction . This quantity is simply the root mean of the squared cross-validated residuals y( (,)) - for i = 1,..., n in the numerator of (20). The cross-... 

The estimates by this approach are veiy much better titan the univariate approaches in this particular example. Figure 5.9 shows the predicted versus known concentrations for pyrene. The root mean square error of prediction is now... 

---