Root-mean-square error of prediction RMSEP

Root-mean-squared error of prediction (RMSEP), 6 50-51... 

The pilot study showed good prospects for predicting crystallization point temperatures directly from the acoustic signatures of the liquid feed into the granulator with an indicated prediction error (root mean square error of prediction) RMSEP = 1.1 °C. 

Chemometrics in Process Analytical Technology (PAT) 409 The main figure of merit in test set validation is the root mean square error of prediction (RMSEP) ... 

NIR models are validated in order to ensure quality in the analytical results obtained in applying the method developed to samples independent of those used in the calibration process. Although constructing the model involves the use of validation techniques that allow some basic characteristics of the model to be established, a set of samples not employed in the calibration process is required for prediction in order to conhrm the goodness of the model. Such samples can be selected from the initial set, and should possess the same properties as those in the calibration set. The quality of the results is assessed in terms of parameters such as the relative standard error of prediction (RSEP) or the root mean square error of prediction (RMSEP). 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The validation set is employed to determine the optimum number of variables to use in the model based on prediction (RMSEP) rather than fit (RMSEO- RM-SEP as a function of the number of variables is plotted in Figure 5.7S for the prediction of the caustic concentration in the validation set, Tlie cuive levels off after three variables and the RMSEP for this model is 0.053 Tliis value is within the requirements of the application (lcr= 0.1) and is not less than the error in the reported concentrations. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The new RMSEP plot in Figure 5-100 is more well behaved than the plot shown in Figure 5-93 (with the incorrect spectrum 3). A minimum is found at 3 factors with a corresponding RMSEP that is almost two orders of magnitude smaller than the minimum in Figure 5-93- The new RMSEP plot shows fairly ideal behavior with a sharp decrease in RMSEP as factors are added and then a slight increase when more than three factors are included. 

Root Mean Square Error of Prediction (RMSEP) (Model Diagnostic) The RMSEP for the determination of caustic is 0.06 wt.% over a range of 7.4-10.4 wt.%. This estimate of prediction ability indicates that the performance of the model is acceptable for the application. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) Prediction error is a useful metric for selecting the optimum number of factors to include in the model. This is because the models are most often used to predict the concentrations in future unknown samples. There are two approaches for generating a validation set for estimating the prediction error internal validation (i.e., cross-validation with the calibration data), or external validation (i.e., perform prediction on a separate validation set). Samples are usually at a premium, and so we most often use a cross- validation approach. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The RMSEP versus number of factors plot in Figure 5.113 shows a break at three factors and a leveling off after six factors. Tlie RMSEP value with six factors (0,04) is comparable to the estimated error in the reported concentrations (0.033), indicating the model is predicting well At this point we tentatively choose a rank six model. The rank three model shows an RMSEP of 0.07 and may well have been considered to be an adequate model, depending on how well the reference values are known. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The RMSEP plot for the MCB model is shown in Figure 5.127. Although the shape of this RMSEP plot is not ideal, it does not exhibit erratic behavior. Tlie first minimum in this plot is at four factors with a lower minimum at six factors. In Section 5.2.1.2, nonlinear behavior was suspected as the root cause of the failure of the DCLS method. Tlicreforc, it is reasonable that a PLS model re-... 

At this point, it is worth noting that the same validation methods that are used to avoid overfitting of quantitative models (Section 8.3.7) can also be used to avoid overfitting in qualitative models. The only difference is that the figure of merit in this case is not the Root Mean Squared Error of Prediction (RMSEP), but rather the percent correct classification or %CC ... 

The b vector chosen by the validation procedure can be employed prospectively to predict concentrations of the analyte of interest in independent data. Similar to the calculation of RMSECV, the root mean square error of prediction (RMSEP) for an independent data set is defined as the square root of the sum of the squares of the differences between predicted and reference concentrations. 

Fig. 7. Distributions of root mean squared errors of prediction (RMSEP) from 1000 test sets (32 samples) randomly selected from the 80 corn samples using full spectra and variables selected by MCUVE and CARS, respectively.

The developed models should be tested using independent samples as validation sets to verify model accuracy and robustness. To evaluate model accuracy, the statistics used were the coefficient of correlation in calibration (rc i), coefficient of correlation in prediction (rpred), root mean square error of calibration (RMSEC), and root mean square error of prediction (RMSEP). 

The total sample sets were separated into calibration set and validation set. Cross validation was first used in calibration sample set to find the optimal principle component number. From figure 4 we can see the best principle component nmnber to be 10 with corresponding highest Rev of 0.91 and lowest RMSEcv of 0.41. Model accuracy was then evaluated on the validation set using the root mean square error of prediction (RMSEP), correlation... 

This method was in fact carried out around two decades ago [30, 31]. However, it was applied only in the fermentation of pure microbial cultures. In a recent report by Acros-Hernandez and coworkers [32], infrared spectroscopy was applied to quantify the PHA produced in microbial mixed cultures. Around 122 spectra from a wide range of production systems were collected and used for calibrating the partial least squares (PLS) model, which relates the spectra with the PHA content (0.03-0.58 w/w) and 3-hydroxyvalerate monomer (0-63 mol%). The calibration models were evaluated by the correlation between the predicted and measured PHA content (R ), root mean square error of calibration, root mean square error of cross validation and root mean square error of prediction (RMSEP). The results revealed that the robust PLS model, when coupled with the Fourier-Transform infrared spectrum, was found to be applicable to predict the PHA content in microbial mixed cultures, with a low RMSEP of 0.023 w/w. This is considered to be a reliable method and robust enough for use in the PHA biosynthesis process using mixed microbial cultures, which is far more complex. 

Finally, a validation set with known concentrations has to be prepared that represents the investigated parameter field. It allows the final evaluation of the calibration quality described by values such as the correlation of predicted and measured concentrations and the root mean square error of prediction (RMSEP), which is comparable to the standard deviation of the predicted values (see Figure 6.6d) [20]. Subsequent to this calibration and validation procedure, a realtime spectroscopic measurement of analyte concentrations within complex reaction mixtures generated in a microreaction process can be accomplished. 

In a multivariate calibration, where a set of NIR spectra (Xnxk, N samples and K variables) is regressed onto a continuous variable (yivxi) such as the fat or moisture content, the statistical errors, the accuracy, are most often used as a quality measure of the calibration. The absolutely most common quality measure of a multivariate calibration is the prediction error, expressed either as root mean square error of prediction (RMSEP) or standard error of performance (SEP). Both are calculated and are the result of a validation process, such as test set or cross-validation. These prediction errors are defined as ... 

Fig. 8.5 PLS validation plots showing for each predicted variable (i.e., sensory descriptor) the root mean squared error of prediction (RMSEP) over the first five model dimensions. RMSEP values were obtained from a leave-one-out bootstrapping algorithm, and both the cross-validated estimate black solid line) and the bias-adjusted eross-validation estimate ned doited line) are shown [38]...

There are three statistics often employed for comparing the performances of multivariate calibration models root mean squared error of calibration (RMSEC), root mean squared error of cross validation (RMSECV), and root mean squared error of prediction (RMSEP). All three methods are based on the calculated root mean squared error (RMSE)... 

---