Chemometrics validation

Multidimensional Data Intercomparisons. Estimation of reliable uncertainty intervals becomes quite complex for non-linear operations and for some of the more sophisticated multidimensional models. For this reason, "chemometric" validation, using common, carefully-constructed test data sets, is of increasing importance. Data evaluation intercomparison exercises are thus analogous to Standard Reference Material (SRM) laboratory intercomparisons, except that the final, data evaluation step of the chemical measurement process is being tested. 

Many people use the term PRESS to refer to the result of leave-one-out cross-validation. This usage is especially common among the community of statisticians. For this reason, the terms PRESS and cross-validation are sometimes used interchangeably. However, there is nothing inate in the definition of PRESS that need restrict it to a particular set of predictions. As a result, many in the chemometrics community use the term PRESS more generally, applying it to predictions other than just those produced during cross-validation. 

L Stable and S. Wold, Partial least square analysis with cross-validation for the two-class problem a Monte Carlo study. J. Chemometrics, 1 (1987) 185-196. 

Kraker, J. J., Hawkins, D. M., Basak, S. C., Natarajan, R., Mills, D. Quantitative structure-activity relationship (QSAR) modebng of juvenile hormone activity Comparison of validation procedures. Chemometr. Intell. Lab. Syst. 2007, 87, 33M2. 

This book is the result of a cooperation between a chemometrician and a statistician. Usually, both sides have quite a different approach to describing statistical methods and applications—the former having a more practical approach and the latter being more formally oriented. The compromise as reflected in this book is hopefully useful for chemometricians, but it may also be useful for scientists and practitioners working in other disciplines—even for statisticians. The principles of multivariate statistical methods are valid, independent of the subject where the data come from. Of course, the focus here is on methods typically used in chemometrics, including techniques that can deal with a large number of variables. Since this book is an introduction, it was necessary to make a selection of the methods and applications that are used nowadays in chemometrics. 

The validity of the results is a central issue, and it is confirmed by comparing traditional methods with their robust counterparts. Robust statistical methods are less common in chemometrics, although they are easy to access and compute quickly. Thus, several robust methods are included. 

The most used resampling strategy in chemometrics to obtain a reasonable large number of predictions is cross validation (CV). CV is also often applied to optimize... 

Dewe, W., Govaerts, B., Boulanger, B., Rozet, E., Chiap, P., Hubert, P. Risk management for analytical methods Conciliating the objectives of the pre-study and in-study validation phases. Chemometr. Intell. Lab. System, 85, 2007, 262-268. 

The chemometric basic tools may be divided into the following typologies of study data exploration, modelling, prediction and validation, design of experiments (DOE), process analytical technology (PAT), quantitative structure-activity relationship (QSAR). Details and relevant literature are reported in the following paragraphs. 

Chemometric quality assurance via laboratory and method intercomparisons of standardized test data sets, finally, is becoming recognized as essential for establishing the validity of detection decisions and estimated detection limits, especially when treating multidimensional data with sophisticated algorithms including several chemical components. 

Wold, S. Eriksson, L. Statistical validation of QSAR results. In Chemometrics Methods in Molecular Design, Waterbeemd, H. v. d. (Ed.). Wiley-VCH, Weinheim, 1995, 309-318. 

This chapter deals with the necessity of representative sampling in the context of PAT. All PAT sensors need to be calibrated with respect to relevant, reliable reference data (Y data). This presupposes that representative samples are at hand for this characterization - but sampled howl Additionally, X signals (X measurements) need to be qualified as representative of the same volume as was extracted for Y characterization, or at least a sufficiently well-matching volume. How does one demonstrate this in a quantitative manner If the quality of both X and Y data involved is suspect, how can a multivariate calibration be expected to be trustworthy This also includes the issue regarding proper validation of the chemometric multivariate calibration(s) involved, which can only be resolved based on proper understanding of the phenomenon of heterogeneity. The TOS delivers answers to all these issues. The TOS constitutes the missing link in PAT. 

Figure 3.10 Hallmark signature of significant sampling bias as revealed in chemometric multivariate calibrations (shown here as a prediction validation). Crab sampling results in an unacceptably high, irreducible RMSEP. While traditionally ascribed to measurement errors, it is overwhelmingly due to ISE.

K.H. Esbensen and L.P. luhus. Representative sampling, data quality, validation - a necessary trinity in chemo-metrics, in Comprehensive Chemometrics, S. Brown, R. Taulor, and B. Walzak (eds), vol. 4, 1-20. Elsevier, Oxford, 2009. 

Many earlier successful PLS prediction models (which in this chapter are presented as examples from industrial production processes) signify that acoustic chemometrics has matured into a proven on-line technology in the PAT domain. It is a salient point here that all were evaluated using test set validation [2]. 

The full-scale industrial experiment demonstrated the feasibility of a convenient, nonintrusive aconstic chemometric facility for reliable ammonia concentration prediction. The training experimental design spanned the industrial concentration range of interest (0-8%). Two-segment cross-validation (test set switch) showed good accnracy (slope 0.96) combined with a satisfactory RMSEP. It is fully possible to further develop this pilot study calibration basis nntil a fnll industrial model has been achieved. There wonld appear to be several types of analogous chemical analytes in other process technological contexts, which may be similarly approached by acoustic chemometrics. 

AU multivariate calibrations must be based on empirical training and validation data sets obtained in fully realistic situations acoustic chemometrics is no exception. Many models are in addition based on indirect multivariate calibration. All industrial applications must always be evaluated only based on test set validation. Reference [2] deals extensively with the merits of the various validation methods, notably when it is admissible, and when not, to use cross-validation. See also Chapters 3 and 12, which give further background for the test set imperative in light of typical material heterogeneity and the Theory of Sampling . 

Chemometrics in Process Analytical Technology (PAT) 409 The main figure of merit in test set validation is the root mean square error of prediction (RMSEP) ... 

Cross-validation methods differ in how the sample subsets are selected for the subvalidation experiments. Several methods that are typically encountered in chemometrics software packages are listed below... 

There is a failure to recognize the plant-site requirements for NIR calibration and validation, snch as the existence of appropriate sampling valves, well-designed sampling protocols, good laboratory reference methods, and variability in the analyte concentrations of interest. More details on these chemometric calibration issues can be found in Chapter 12. 

Monitoring the MMA/DMAAm reaction is challenging becanse both monomers have very similar NIR spectra, and because other interfering snbstances are present in the reaction mixtnre. The anthors prepared calibration samples gravimetrically and made the NIR measurements at reaction temperatnres. The calibration sets consisted of only five or six samples, which is considerably fewer than standard recommendations for NIR chemometric model development. The final models for MMA and DMAAm were validated with an internal validation set as well as an external reaction validation. The performance of the models is summarized in Table 15.6. This table inclndes a measurement of the standard deviation of an external GC method... 

All of these studies suffer from the fact that they were carried out on relatively small datasets of more or less homogeneous polymers and are generally not well validated. As such, they indicate that there may be useful chemometric methods here, but there is considerable scope for further studies on much larger and heterogeneous sample sets to demonstrate general applicability and usefulness. 

Although the chemometric tools discussed in this book into both of these categories, most of the emphasis is on implicit modeling. Valid explicit models are notnmunon in practice and, therefore, implicit models are often necessary to anfee the data and/or construct predictive models. 

As the definition says, a model is a description of a real phenomenon performed by means of mathematical relationships (Box and Draper, 1987). It follows that a model is not the reality itself it is just a simplified representation of reality. Chemometric models, different from the models developed within other chemical disciplines (such as theoretical chemistry and, more generally, physical chemistry), are characterized by an elevated simplicity grade and, for this reason, their validity is often limited to restricted ranges of the whole experimental domain. 

Nevertheless, chemometric models are not developed with the aim of supporting a theory or describing a phenomenon from a general point of view, but with the aim of obtaining answers for particular real problems. Therefore, if the validity range of a model corresponds to the region of... 

J.A. Van Leeuwen, L.M.C. Buydens, B.G.M. Vandeginste, G. Kateman, P.J. Schoenmakers, M. Mulholland, RES, an expert system for the set-up and interpretation of a ruggedness test in HPLC method validation. Part 1 The ruggedness test in HPLC method validatioa Chemometrics and Intelligent Laboratory systems, 10 (1991) 337-347. 

---