Calibration data modeling

Figure 4-230 shows the photograph of a Develco high-temperature directional sensor. For all the sensor packages, calibration data taken at 25, 75, 125, 150, 175 and 200°C are provided. Computer modeling coefficients provide sensor accuracy of 0.001 G and 0.1° alignment from 0 to 175°C. From 175 to 200°C the sensor accuracy is 0.003 G and 0.1° alignment. 

The ultimate goal of multivariate calibration is the indirect determination of a property of interest (y) by measuring predictor variables (X) only. Therefore, an adequate description of the calibration data is not sufficient the model should be generalizable to future observations. The optimum extent to which this is possible has to be assessed carefully when the calibration model chosen is too simple (underfitting) systematic errors are introduced, when it is too complex (oveifitting) large random errors may result (c/. Section 10.3.4). 

Let us assume that we have collected a set of calibration data (X, Y), where the matrix X (nxp) contains the p > 1 predictor variables (columns) measured for each of n samples (rows). The data matrix Y (nxq) contains the q variables which depend on the X-data. The general model in calibration reads... 

The advantage of the inverse calibration approach is that we do not have to know all the information on possible constituents, analytes of interest and inter-ferents alike. Nor do we need pure spectra, or enough calibration standards to determine those. The columns of C (and P) only refer to the analytes of interest. Thus, the method can work in principle when unknown chemical interferents are present. It is of utmost importance then that such interferents are present in the Ccdibration samples. A good prediction model can only be derived from calibration data that are representative for the samples to be measured in the future. 

We chose the number of PCs in the PCR calibration model rather casually. It is, however, one of the most consequential decisions to be made during modelling. One should take great care not to overfit, i.e. using too many PCs. When all PCs are used one can fit exactly all measured X-contents in the calibration set. Perfect as it may look, it is disastrous for future prediction. All random errors in the calibration set and all interfering phenomena have been described exactly for the calibration set and have become part of the predictive model. However, all one needs is a description of the systematic variation in the calibration data, not the... 

The complete data series is used to calculate the temperature response, but only certain parts of the experimental data are used to calculate the error. An example of a calibration run is given in Figure 53, the final calibrated TRNSYS model run is shown in Figure 54. Using the first part of the data (with constant heat flux) an estimate of ground thermal conductivity of 2.15 was obtained. Yavatzturk s method yielded an estimate of 2.18, while the estimate obtained with the TRNSYS parameter estimation method was 2.10. 

Some models require calibration to optimize input parameters they are best used in a research setting where it is possible to make measurements with which to calibrate the model for a particular site. Appropriate measured hydrologic data are seldom available to calibrate a model for a particular landfill site. Therefore, engineering models used for ET cover design should not require calibration. 

Draper and Smith [1] discuss the application of DW to the analysis of residuals from a calibration their discussion is based on the fundamental work of Durbin, et al in the references listed at the beginning of this chapter. While we cannot reproduce their entire discussion here, at the heart of it is the fact that there are many kinds of serial correlation, including linear, quadratic and higher order. As Draper and Smith show (on p. 64), the linear correlation between the residuals from the calibration data and the predicted values from that calibration model is zero. Therefore if the sample data is ordered according to the analyte values predicted from the calibration model, a statistically significant value of the Durbin-Watson statistic for the residuals in indicative of high-order serial correlation, that is nonlinearity. 

Yapo PO, Gupta HV, Sorooshian S (1996) Automatic calibration of conceptual rainfall-runoff models sensitivity to calibration data. J Hydrol 181(1—4) 23—48... 

All regression methods aim at the minimization of residuals, for instance minimization of the sum of the squared residuals. It is essential to focus on minimal prediction errors for new cases—the test set—but not (only) for the calibration set from which the model has been created. It is relatively easy to create a model— especially with many variables and eventually nonlinear features—that very well fits the calibration data however, it may be useless for new cases. This effect of overfitting is a crucial topic in model creation. Definition of appropriate criteria for the performance of regression models is not trivial. About a dozen different criteria— sometimes under different names—are used in chemometrics, and some others are waiting in the statistical literature for being detected by chemometricians a basic treatment of the criteria and the methods how to estimate them is given in Section 4.2. 

Note that the described model best fits the given (calibration) data, but is not necessarily optimal for predictions (see Section 4.2). 

FIGURE 4.24 PLS as a multiple linear regression method for prediction of a property y from variables xi,..., xm, applying regression coefficients b1,...,bm (mean-centered data). From a calibration set, the PLS model is created and applied to the calibration data and to test data. 

Treatment of a real, imperfect calibration data set revealed the full complexity and breadth of the calibration curve -detection limit problem, ranging from varying statistical weights to an uncertain model and data containing possible blunders to an artificially imposed response threshold. 

In the text which follows we shall examine in numerical detail the decision levels and detection limits for the Fenval-erate calibration data set ( set-B ) provided by D. Kurtz (17). In order to calculate said detection limits it was necessary to assign and fit models both to the variance as a function of concentration and the response (i.e., calibration curve) as a function of concentration. No simple model (2, 3 parameter) was found that was consistent with the empirical calibration curve and the replication error, so several alternative simple functions were used to illustrate the approach for calibration curve detection limits. A more appropriate treatment would require a new design including real blanks and Fenvalerate standards spanning the region from zero to a few times the detection limit. Detailed calculations are given in the Appendix and summarized in Table V. 

The mathematical model may not closely fit the data. For example. Figure 1 shows calibration data for the determination of iron in water by atomic absorption spectrometry (AAS). At low concentrations the curve is first- order, at high concentrations it is approximately second- order. Neither model adequately fits the whole range. Figure 2 shows the effects of blindly fitting inappropriate mathematical models to such data. In this case, a manually plotted curve would be better than either a first- or second-order model. 

Improved mathematical models. First or second order linear equations adequately fit much calibration data. If neither model is appropriate, the following semi-empirical multiple curve procedure may be used. 

There are a number of ways to model calibration data by regression. Host researchers have attempted to describe data with a linear function. Others ( 4,5 ) have chosen a higher order or a polynomial method. One report ( 6 ) compared the error in the interpolation using linear segments over a curved region verses using a curvilinear regression. Still others ( 7,8 ) chose empirical or spline functions. Mixed model descriptions have also been used ( 4,7 ). 

Raman spectroscopy s sensitivity to the local molecular enviromnent means that it can be correlated to other material properties besides concentration, such as polymorph form, particle size, or polymer crystallinity. This is a powerful advantage, but it can complicate the development and interpretation of calibration models. For example, if a model is built to predict composition, it can appear to fail if the sample particle size distribution does not match what was used in the calibration set. Some models that appear to fail in the field may actually reflect a change in some aspect of the sample that was not sufficiently varied or represented in the calibration set. It is important to identify any differences between laboratory and plant conditions and perform a series of experiments to test the impact of those factors on the spectra and thus the field robustness of any models. This applies not only to physical parameters like flow rate, turbulence, particulates, temperature, crystal size and shape, and pressure, but also to the presence and concentration of minor constituents and expected contaminants. The significance of some of these parameters may be related to the volume of material probed, so factors that are significant in a microspectroscopy mode may not be when using a WAl probe or transmission mode. Regardless, the large calibration data sets required to address these variables can be burdensome. 

Conceptually, the value for a given sample reflects the extremeness of that sample s response within the PCA model space, whereas the Q valne reflects the amonnt of the sample s response that is outside of the PCA model space. Therefore, both metrics are necessary to fnlly assess the abnormality of a response. In practice, before one can nse a PCA model as a monitor, one mnst set a confidence limit on each of these metrics. There are several methods for determining these confidence limits [30,31], bnt these nsually require two sets of information (1) the set of and Q values that are obtained when the calibration data (or a suitable set of independent test data) is applied to the PCA model, and (2) a user-specified level of confidence (e.g. 95%, 99%, or 99.999%). Of conrse, the latter is totally at the discretion of the nser, and is driven by the desired sensitivity and specificity of the monitoring application. 

To provide calibration data that can be used to model or assess any nonlinear effects in the analyzer data. 

For the styrene-butadiene example, the use of the PCR method to develop a calibration for di-butadiene is summarized in Table 12.6. It should be mentioned that the data were mean-centered before application of the PCR method. Figure 12.12 shows the percentage of explained variance in both x (the spectral data) andy (the c/i-butadiene concentration data) after each principal component. After four principal components, it does not appear that the use of any additional PCs results in a large increase in the explained variance of X or y. If a PCR regression model using four PCs is built and applied to the calibration data, a fit RMSEE of 1.26 is obtained. 

However, the PLS-DA method requires sufficient calibration samples for each class to enable effective determination of discriminant thresholds, and one must be very careful to avoid overfitting of a PLS-DA model through the use of too many PLS factors. Also, the PLS-DA method does not explicitly account for response variations within classes. Although such variations in the calibration data can be useful information for assigning and determining uncertainty in class assignments during prediction, it will be treated essentially... 

---