Fitting error—in-variables models

Valko, P. and S. Vajda, "An Extended Marquardt -type Procedure for Fitting Error-In-Variables Models", Computers Chem. Eng., 11, 37-43 (1987). 

Valko, P., and Vadja, S. (1987). An extended Marquardt-type procedure for fitting error-in-variable models. Comput. Chem. Eng. 11, 37-43. 

M52 Fitting an error-in-variables model of the form F(Z,P)=0 modified Patino-Leal - Reilly method 5200 5460... 

In order to assess the optimal complexity of a model, the RMSEP statistics for a series of different models with different complexity can be compared. In the case of PLS models, it is most common to plot the RMSEP as a function of the number of latent variables in the PLS model. In the styrene—butadiene copolymer example, an external validation set of 7 samples was extracted from the data set, and the remaining 63 samples were used to build a series of PLS models for ris-butadicne with 1 to 10 latent variables. These models were then used to predict the ris-butadicne of the seven samples in the external validation set. Figure 8.19 shows both the calibration fit error (in RMSEE) and the validation prediction error (RMSEP) as a function of the number of... 

Sheiner and Beal (1981) have pointed out the errors involved in using the correlation coefficient to assess the goodness of fit (GOF) in pharmacokinetic models. Pearson s correlation coefficient overestimates the predictability of the model because it represents the best linear line between two variables. A more appropriate estimator would be a measure of the deviation from the line of unity because if a model perfectly predicts the observed data then all the predicted values should be equal to all the observed values and a scatter plot of observed vs. predicted values should form a straight line whose origin is at the point (0,0) and whose slope is equal to a 45° line. Any deviation from this line represents both random and systemic error. 

For each experiment, the true values of the measured variables are related by one or more constraints. Because the number of data points exceeds the number of parameters to be estimated, all constraint equations are not exactly satisfied for all experimental measurements. Exact agreement between theory and experiment is not achieved due to random and systematic errors in the data and to "lack of fit" of the model to the data. Optimum parameters and true values corresponding to the experimental measurements must be found by satisfaction of an appropriate statistical criterion. 

This sum, when divided by the number of data points minus the number of degrees of freedom, approximates the overall variance of errors. It is a measure of the overall fit of the equation to the data. Thus, two different models with the same number of adjustable parameters yield different values for this variance when fit to the same data with the same estimated standard errors in the measured variables. Similarly, the same model, fit to different sets of data, yields different values for the overall variance. The differences in these variances are the basis for many standard statistical tests for model and data comparison. Such statistical tests are discussed in detail by Crow et al. (1960) and Brownlee (1965). 

Once we have estimated the unknown parameters that appear in an algebraic or ODE model, it is quite important to perform a few additional calculations to establish estimates of the standard error in the parameters and in the expected response variables. These additional computational steps are very valuable as they provide us with a quantitative measure of the quality of the overall fit and inform us how trustworthy the parameter estimates are. 

The following criteria are usually directly applied to the calibration set to enable a fast comparison of many models as it is necessary in variable selection. The criteria characterize the fit and therefore the (usually only few) resulting models have to be tested carefully for their prediction performance for new cases. The measures are reliable only if the model assumptions are fulfilled (independent normally distributed errors). They can be used to select an appropriate model by comparing the measures for models with various values of in. 

An important point is the evaluation of the models. While most methods select the best model at the basis of a criterion like adjusted R2, AIC, BIC, or Mallow s Cp (see Section 4.2.4), the resulting optimal model must not necessarily be optimal for prediction. These criteria take into consideration the residual sum of squared errors (RSS), and they penalize for a larger number of variables in the model. However, selection of the final best model has to be based on an appropriate evaluation scheme and on an appropriate performance measure for the prediction of new cases. A final model selection based on fit-criteria (as mostly used in variable selection) is not acceptable. 

Root Mean Square Error of Prediction (RMSEP) Plot (Model Diagnostic) The validation set is employed to determine the optimum number of variables to use in the model based on prediction (RMSEP) rather than fit (RMSEO- RM-SEP as a function of the number of variables is plotted in Figure 5.7S for the prediction of the caustic concentration in the validation set, Tlie cuive levels off after three variables and the RMSEP for this model is 0.053 Tliis value is within the requirements of the application (lcr= 0.1) and is not less than the error in the reported concentrations. 

For the example data, the RMSEC is calculated for models containing 1 -22 variables (adding the variables in the order listed in Table 5.9). The RMSEC versus the number of variables included in the model is plotted in Figure 5.66 for component A. nte fit improves as variables are added to the model (RMSEC decreases). However, knowing these results reflect model fit, there is a concern about overfitting (i.e., fitting noise from the calibration data). It is known that the error in the concentration values is 0.010 (la). The RMSEC drops below this level after the fifth variable is included, and therefore the tentative conclusion is that a four-variable model is appropriate. 

The van Laar parameters A = 5135.9 J/mol and = 43213.7 J/mol yield a good fit. The observed variables are only slightly corrected to satisfy the model equations. The quantity "equation error after correction" is expressed in Pascals, hence the above values are negligible small. 

ANOVA of the data confirms that there is a statistically significant relationship between the variables at the 99% confidence level. The i -squared statistic indicates that the model as fitted explains 96.2% of the variability. The adjusted P-squared statistic, which is more suitable for comparing models with different numbers of independent variables, is 94.2%. The prediction error of the model is less than 10%. Results of this protocol are displayed in Table 18.1. Validation results of the regression model are displayed in Table 18.2. 

Residual standard deviation — (experimental data corresponding to the dependent variable Y and a respective array of values calculated according to a given mathematical model F(x) . Consequently, it is assumed that the dependent variable contains all the error of measurements and there is no error in the values of the independent variable V. A very low standard deviation of a fit does not mean that the assigned mathematical model is reasonable since the parameters resulting of a fit may have values that make no sense, or the proposed model can be a bad descriptor of the experiment. If each data point provides equally precise information about the total process variation, the value of crr is defined as the positive square root of the sum of the squares of the -> residuals, divided by the number of experimental data points ri, minus the number of fitted parameters p [i, ii]. It can be calculated as ... 

The B score (Brideau et al., 2003) is a robust analog of the Z score after median polish it is more resistant to outliers and also more robust to row- and column-position related systematic errors (Table 14.1). The iterative median polish procedure followed by a smoothing algorithm over nearby plates is used to compute estimates for row and column (in addition to plate) effects that are subtracted from the measured value and then divided by the median absolute deviation (MAD) of the corrected measures to robustly standardize for the plate-to-plate variability of random noise. A similar approach uses a robust linear model to obtain robust estimates of row and column effects. After adjustment, the corrected measures are standardized by the scale estimate of the robust linear model fit to generate a Z statistic referred to as the R score (Wu, Liu, and Sui, 2008). In a related approach to detect and eliminate systematic position-dependent errors, the distribution of Z score-normalized data for each well position over a screening run or subset is fitted to a statistical model as a function of the plate the resulting trend is used to correct the data (Makarenkov et al., 2007). 

Although classical calibration is widely used, it is not always the most appropriate approach in chemistry, for two main reasons. First, the ultimate aim is usually to predict the concentration (or independent variable) from the spectrum or chromatogram (response) rather than vice versa. The second relates to error distributions. The errors in the response are often due to instrumental performance. Over the years, instruments have become more reproducible. The independent variable (often concentration) is usually determined by weighings, dilutions and so on, and is often by far the largest source of errors. The quality of volumetric flasks, syringes and so on has not improved dramatically over the years, whereas the sensitivity and reproducibility of instruments has increased manyfold. Classical calibration fits a model so diat all errors are in the response [Figure 5.4(a)], whereas a more appropriate assumption is that errors are primarily in the measurement of concentration [Figure 5.4(b)]. 

---