Validation procedure, regression

In another work, Parra and coworkers proposed a method based on chemically modified voltammetric electrodes for the identification of adulterations made in wine samples, by addition of a number of forbidden adulterants frequently used in the wine industry to improve the organoleptic characteristics of wines, like, for example, tartaric acid, tannic acid, sucrose, and acetaldehyde (Parra et ah, 2006b). The patterns identified via PCA allowed an efficient detection of the wine samples that had been artificially modified. In the same study, PLS regression was applied for a quantitative prediction of the substances added. Model performances were evaluated by means of a cross-validation procedure. 

S. Lanteri, Full validation procedures for feature selection in classification and regression problems, Chemom. Intell. Lab. Syst., 15, 1992, 159-169. 

Sometimes the question arises whether it is possible to find an optimum regression model by a feature selection procedure. The usual way is to select the model which gives the minimum predictive residual error sum of squares, PRESS (see Section 5.7.2) from a series of calibration sets. Commonly these series are created by so-called cross-validation procedures applied to one and the same set of calibration experiments. In the same way PRESS may be calculated for a different sets of features, which enables one to find the optimum set . 

The results with PCR and PLS regression include the number of PCs obtained by leave-one-out cross-validation procedure, the values of regression coefficients for X variables, the value of R, and the root mean square error of calibration (RMSE C ) and the root mean square error of prediction by cross-validation proce-... 

Lanteri, S. (1992). Full Validation Procedures for Feature Selection in Classification and Regression Problems. Chemom.lntellLab.Syst., 15,159-169. 

Scheme I Schematic representation of (a) PLS model (b) UVE-PLS model and (c) matrix B, containing regression coefficients calculated by the leave-one-out cross-validation procedure and their mean values, standard deviations, and stability.

Indicator variables (I 1.0/0.0) are used to code the presence or absence of a key substructure. Regression of real numbers (pIC50 s) against a matrix of indicator variables is a valid procedure for large sets, as in the Free-Wllson method. However, many of the sets in this study are small (n = 7-10) and it is probable that statistical measures for these sets are only approximate. The overall consistency of substructure dependence in both small and larger sets is considered to validate these measures in a seml-quantltatlve sense. 

Aires-de-Sousa and Gasteiger used four regression techniques [multiple linear regression, perceptron (a MLF ANN with no hidden layer), MLF ANN, and v-SVM regression] to obtain a quantitative structure-enantioselectivity relationship (QSER). The QSER models the enantiomeric excess in the addition of diethyl zinc to benzaldehyde in the presence of a racemic catalyst and an enan-tiopure chiral additive. A total of 65 reactions constituted the dataset. Using 11 chiral codes as model input and a three-fold cross-validation procedure, a neural network with two hidden neurons gave the best predictions ANN 2 hidden neurons, R pred = 0.923 ANN 1 hidden neurons, R pred = 0.906 perceptron, R pred = 0.845 MLR, R p .d = 0.776 and v-SVM regression with RBF kernel, R pred = 0.748. 

The second task discussed is the validation of the regression models with the aid of the cross-validation (CV) procedures. The leave-one-out (LOO) as well as the leave-many-out CV methods are used to evaluate the prognostic possibilities of QSAR. In the case of noisy and/or heterogeneous data the LM method is shown to exceed sufficiently the LS one with respect to the suitability of the regression models built. The especially noticeable distinctions between the LS and LM methods are demonstrated with the use of the LOO CV criterion. 

Valid physical property relationships form an important feature of a process model. To validate a model, representative data must fit by some type of correlation using an optimization technique. Nonlinear regression instead of linear regression may be involved in the fitting. We illustrate the procedure in this example. 

For partial least-squares (PLS) or principal component regression (PCR), the infrared spectra were transferred to a DEC VAX 11/750 computer via the NIC-COM software package from Nicolet. This package also provided utility routines used to put the spectra into files compatible with the PLS and PCR software. The PLS and PCR program with cross-validation was provided by David Haaland of Sandia National Laboratory. A detailed description of the program and the procedures used in it has been given (5). 

---