Chemometrics cross validation

Many people use the term PRESS to refer to the result of leave-one-out cross-validation. This usage is especially common among the community of statisticians. For this reason, the terms PRESS and cross-validation are sometimes used interchangeably. However, there is nothing inate in the definition of PRESS that need restrict it to a particular set of predictions. As a result, many in the chemometrics community use the term PRESS more generally, applying it to predictions other than just those produced during cross-validation. 

L Stable and S. Wold, Partial least square analysis with cross-validation for the two-class problem a Monte Carlo study. J. Chemometrics, 1 (1987) 185-196. 

The most used resampling strategy in chemometrics to obtain a reasonable large number of predictions is cross validation (CV). CV is also often applied to optimize... 

The full-scale industrial experiment demonstrated the feasibility of a convenient, nonintrusive aconstic chemometric facility for reliable ammonia concentration prediction. The training experimental design spanned the industrial concentration range of interest (0-8%). Two-segment cross-validation (test set switch) showed good accnracy (slope 0.96) combined with a satisfactory RMSEP. It is fully possible to further develop this pilot study calibration basis nntil a fnll industrial model has been achieved. There wonld appear to be several types of analogous chemical analytes in other process technological contexts, which may be similarly approached by acoustic chemometrics. 

AU multivariate calibrations must be based on empirical training and validation data sets obtained in fully realistic situations acoustic chemometrics is no exception. Many models are in addition based on indirect multivariate calibration. All industrial applications must always be evaluated only based on test set validation. Reference [2] deals extensively with the merits of the various validation methods, notably when it is admissible, and when not, to use cross-validation. See also Chapters 3 and 12, which give further background for the test set imperative in light of typical material heterogeneity and the Theory of Sampling . 

Cross-validation methods differ in how the sample subsets are selected for the subvalidation experiments. Several methods that are typically encountered in chemometrics software packages are listed below... 

An important chemometric tool is called cross-validation. The basis of the method is that the predictive ability of a model formed on part of a dataset can be tested out by how well it predicts the remainder of the data. 

Louwerse DJ, Kiers HAL, Smilde AK, Cross-validation of multiway component models, Journal of Chemometrics, 1999, 13, 491-510. 

Osten DW, Selection of optimal regression models via cross-validation, Journal of Chemometrics, 1988,2, 39-48. 

Scarponi G, Moret I, Capodaglio G, Romanazzi M, Cross-validation, influential observations and selection of variables in chemometric studies of wines by principal component analysis, Journal of Chemometrics, 1990,4,217-240. 

Partial least squares Partial least squares (PLS) is a statistical technique often applied to relate physicochemical properties to one or several measurements of biological activity. The PLS results consist of two sets of computed factors which are, on the one hand, linear combinations of the chemical descriptors and, on the other hand, linear combinations of the biological activities. Partial least squares finds many applications in chemometrics and, e.g., in the Tripos CoMFA approach. Normally used in conjunction with cross-validation. 

W. Wu, D.L. Massart and S. de Jong, The Kernel PCA Algorithms for Wide Data. Part II Fast Cross-Validation and Application in Classification of NIR Data, Chemometrics and Intelligent Laboratory Systems. 37 (1997) 271-280. 

Model comparison plays a central role in statistical learning and chemometrics. Performances of models need to be assessed using a given criterion based on which models can be compared. To our knowledge, there exist a variety of criteria that can be applied for model assessment, such as Akaike s information criterion (AIC) [1], Bayesian information criterion (BIC) [2], deviance information criterion (DlC),Mallow s Cp statistic, cross validation [3-6] and so on. There is a large body of literature that is devoted to these criteria. With the aid of a chosen criterion, different models can be comp>ared. For example, a model with a smaller AIC or BIC is preferred if AIC or BIC are chosen for model assessment. 

P. Filzmoser, B. Liebmann, K. Varmuza, Repeated double cross validation, J Chemometr,... 

Kvalheim, O.M. Karstang, T.V. (1992). SIMCA-Classification by means of disjoint cross validated principal component models. In Multivariate Pattern Recognition in Chemometrics, illustrated by case studies, R.G. Brereton (Ed.), 209-245, Elsevier, ISBN 0444897844, Amsterdam, Netherland... 

Another example of applying chemometrics to separations data is depicted in Figures 8 and 9. Here, interval PLS (iPLS) was applied to blends of oils in order to quantify the relative concentration of olive oil in the samples (de la Mata-Espinosa et al., 2011b). iPLS divides the data into a number of intervals and then calculates a PLS model for each interval. In this example, the two peak segments which presented the lower root mean square error of cross validation (RMSECV) were used for building the final PLS model. 

Spectrophotometric monitoring with the aid of chemometrics has also been applied to more complex mixtures. To solve the mixtures of corticosteroid de-xamethasone sodium phosphate and vitamins Bg and Bi2, the method involves multivariate calibration with the aid of partial least-squares regression. The model is evaluated by cross-validation on a number of synthetic mixtures. The compensation method and orthogonal function and difference spectrophotometry are applied to the direct determination of omeprazole, lansoprazole, and pantoprazole in grastroresistant formulations. Inverse least squares and PCA techniques are proposed for the spectrophotometric analyses of metamizol, acetaminophen, and caffeine, without prior separation. Ternary and quaternary mixtures have also been solved using iterative algorithms. 

By some classical chemometric methods such as Fisher method, the samples of official Ru wares can be also clearly separated from those of folk Ru wares based on the trace element contents of porcelain body. Their projection map is shown in Fig. 11.4. But by cross validation there are some wrong prediction results. It implies that there are some overfitting effects. 

Camacho J, Ferrer A. Cross-validation in PCA models with the element-wise k-fold (ekf) algorithm theoretical aspects. J Chemometr 2012 26 361-73. 

Xu Q-S, Liang Y-Z. Monte Carlo cross validation. Chemometr Intell Lab Syst 2001 56 1-11. 

Some technical aspects of developing chemometric models of spectroscopic data have been discussed, particularly the importance of preprocessing data properly to obtain meaningful multivariate models. Further, the importance of implementing cross validation procedures to ensure that especially supervised models are capable of making valid predictions has been discussed. 

Andersen E, Dyrstad K, Westad F, Martens H. Reducing over-optimism in variable selection by cross-model validation. Chemometr Intell Lab Syst 2006 84 69-74. 

---