RMSEP

The Standard Error of Prediction (SEP) is supposed to refer uniquely to those situations when a calibration is generated with one data set and evaluated for its predictive performance with an independent data set. Unfortunately, there are times when the term SEP is wrongly applied to the errors in predicting y variables of the same data set which was used to generate the calibration. Thus, when we encounter the term SEP, it is important to examine the context in order to verify that the term is being used correctly. SEP is simply the square root of the Variance of Prediction, s2. The RMSEP (see below) is sometimes wrongly called the SEP. Fortunately, the difference between the two is usually negligible. 

The Root Mean Standard Error of Prediction (RMSEP) is simply the square root of the MSEP. The RMSEP is sometimes wrongly called the SEP. Fortunately, the difference between the two is usually negligible. 

The experimental errors on the %DE measurements are estimated to be between 1 and 2 %, taking into account a relative long time span and the involvement of different lab-workers. As indicated by Table 2 the best models converge to an RMSEP of 1.5 % to refine the models further the experimental chemical errors have to be thoroughly investigated. 

Root-mean-squared error of prediction (RMSEP), 6 50-51... 

Unfortunately, definitions, nomenclature, and abbreviations used for performance criteria are sometimes confusing (Frank and Todeschini 1994 Kramer 1998). For instance in the abbreviations MSEC, PRESS, RMSEP, SEC, SEE, SEP, E means error or estimate, R means residual or root, and S means squared or standard or sum. To make it not too complicated, at least in these examples, C is always calibration, M is mean, and P is prediction. 

The root mean squared error (RMSE) is the square root of MSE, and can again be given for calibration (RMSEC or RMSECAL), CV (RMSECV or RMSECv) or for prediction/test (RMSEP, or RMSEtest). In the case of a negligible bias, RMSEP and SEP are almost identical, as well as MSEniST and SEP2. 

Figure 3.10 Hallmark signature of significant sampling bias as revealed in chemometric multivariate calibrations (shown here as a prediction validation). Crab sampling results in an unacceptably high, irreducible RMSEP. While traditionally ascribed to measurement errors, it is overwhelmingly due to ISE.

The pilot study showed good prospects for predicting crystallization point temperatures directly from the acoustic signatures of the liquid feed into the granulator with an indicated prediction error (root mean square error of prediction) RMSEP = 1.1 °C. 

The results displayed in Figure 9.10 show that it is possible to calibrate a model for the average particle size (slope 0.96, relative RMSEP = 3%), which would serve as a good indication to the process control operators to optimize the process to produce granules within product specifications. Test samples still have to be sent to the laboratory with some regularity to produce more accurate meastires, but much less frequently... 

In spite of the fact that this model has a relatively large RMSEP of 14%, the general potential to pick up the important production trend changes is at an acceptable level for improved process monitoring. 

Figure 9.23 Prediction resuits for ammonia, validated with two-segment cross validation (test set switch). Slope = 0.96. RMSEP = 0.48% ammonia.

The full-scale industrial experiment demonstrated the feasibility of a convenient, nonintrusive aconstic chemometric facility for reliable ammonia concentration prediction. The training experimental design spanned the industrial concentration range of interest (0-8%). Two-segment cross-validation (test set switch) showed good accnracy (slope 0.96) combined with a satisfactory RMSEP. It is fully possible to further develop this pilot study calibration basis nntil a fnll industrial model has been achieved. There wonld appear to be several types of analogous chemical analytes in other process technological contexts, which may be similarly approached by acoustic chemometrics. 

Chemometrics in Process Analytical Technology (PAT) 409 The main figure of merit in test set validation is the root mean square error of prediction (RMSEP) ... 

Figure 12.26 Plot of the calibration error (RMSEE) and the validation error (RMSEP) as a function of the number of latent variables, for the case where 63 of the styrene-butadiene copolymer samples were selected for calibration, and the remaining seven samples were used for validation.

After all of the subvalidations are performed, the RMSEP metrics obtained from each of them are combined... 

NIR models are validated in order to ensure quality in the analytical results obtained in applying the method developed to samples independent of those used in the calibration process. Although constructing the model involves the use of validation techniques that allow some basic characteristics of the model to be established, a set of samples not employed in the calibration process is required for prediction in order to conhrm the goodness of the model. Such samples can be selected from the initial set, and should possess the same properties as those in the calibration set. The quality of the results is assessed in terms of parameters such as the relative standard error of prediction (RSEP) or the root mean square error of prediction (RMSEP). 

Parameter Units Concentration range Wavelength range (nm) Number of PLSl factors Regression coefficient RMSEC RMSEP... 

Monomer RMSEC (mol/L) RMSEP (mol/L) internal validation RMSEP (mol/L) external validation GC STD (mol/L)... 

PbSe lead selenide RMSEP root-mean-squared error of... 

Root Mem Square Error of Prediction (RMSEP) (Model Diagnostic) The RMSEP is anciaer diagnostic for examining the errors in the predicted concentrations. Whie the statistical prediction error discussed earlier quantifies preci-... 

The RMSEP characterizes both the accuracy and precision errors expected for future predictions. 

When the model is performing well, the RMSEP should be comparable to the known error in the concentration reference method. 

Root Mea Square Error of Prediction (RMSEP) (Model Diagnostic) The RMSEP values for all four components are numerically summarized in Table 5.6. They are large owing to the bias in the predictions. Several reasons for this bias can be proposed, including an inaccurate reference method, transcription errcKS, poor experimental procedures, changes in densiw and/or pathlength, l t scatter in the instrument or sample, chemical interactions,... 

The RMSEP plotted versus the number of variables included in the model for componem A indicates that the optimum model contains one variable (see Figure 5-67). The prediction error with one variable (0.016) is greater than the known coneersration error of 0.010, which 1 an indication that tiic niodd is not overfittii. Assuming the model is correct, the fact that the RMSEP is greater than e errors in the known concentrations is due to errors in R. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The validation set is employed to determine the optimum number of variables to use in the model based on prediction (RMSEP) rather than fit (RMSEO- RM-SEP as a function of the number of variables is plotted in Figure 5.7S for the prediction of the caustic concentration in the validation set, Tlie cuive levels off after three variables and the RMSEP for this model is 0.053 Tliis value is within the requirements of the application (lcr= 0.1) and is not less than the error in the reported concentrations. 

To select m optimum rank, the RMSEP is evaluated with models constructed using different numbers of factors. The RMSEP decreases when the predicted valu (c.) is close to the known value (cp and, therefore, small RMSEP values are desired (see Equation 5-36). To choose the optimum number of factors,.2plot of RMSEP versus the number of factors is examined. This plot typically Resents results from models constructed using more factors than are expecasd to be significant. 

Figure 5-92cshows an RMSEP plot diat displays erratic behavior. This type of plot is obseised when the algorithm is not able to model the concentration variations. It casalso result when gross errors are present in the reference values (e.g., transcaption errors in the concentration values, mixed up samples, and/or poor r ence methods). 

---