Regression Least-Squares Classification

C.H. Yeh and C.H. Spiegelman, Partial least squares and classification and regression trees. Chemom. and Intell. Lab. Systems, 22 (1994) 17-23. 

Partial least square (PLS) regression model describes the dependences between two variables blocks, e.g. sensor responses and time variables. Let the X matrix represent the sensor responses and the Y matrix represent time, the X and Y matrices could be approximated to few orthogonal score vectors, respectively. These components are then rotated in order to get as good a prediction of y variables as possible [25], Linear discriminant analysis (LDA) is among the most used classification techniques. The method maximises the variance between... 

Canonical Correlation Analysis Principal Component Regressionb Classification and Regression Trees (CART) Linear Learning Machine Neural Networks Adaptive Least Squares Genetic Programming Logistic Regression... 

The methods of data analysis depend on the nature of the final output. If the problem is one of classification, a number of multivariate classifiers are available such as those based on principal components analysis (SIMCA), cluster analysis and discriminant analysis, or non-linear artificial neural networks. If the required output is a continuous variable, such as a concentration, then partial least squares regression or principal component regression are often used [20]. 

Equations (25) are linear with respect to x and this classification technique is referred to as /inear discriminant analysis, with the discriminant function obtained by least squares analysis, analogous to multiple regression analysis. 

Abbreviations used sigmoidal regression (SR), classification and regression trees (CART), partial least square projection to latent structure... 

Besides the classical Discriminant Analysis (DA) and the k-Nearest Neighbor (k-NN), other classification methods widely used in QSAR/QSPR studies are SIMCA, Linear Vector Quantization (LVQ), Partial Least Squares-Discriminant Analysis (PLS-DA), Classification and Regression Trees (CART), and Cluster Significance Analysis (CSA), specifically proposed for asymmetric classification in QSAR. 

Data arrays (one-way data) consist of spectral values taken at a given time point or electrophoretic responses at a given wavelength over time. One-way data can be used for sample characterization, classification, and quantification with multivariate calibration methods (e.g., principal component analysis and partial least square regression). 

To establish a correlation between the concentrations of different kinds of nucleosides in a complex metabolic system and normal or abnormal states of human bodies, computer-aided pattern recognition methods are required (15, 16). Different kinds of pattern recognition methods based on multivariate data analysis such as principal component analysis (PCA) (8), partial least squares (16), stepwise discriminant analysis, and canonical discriminant analysis (10, 11) have been reported. Linear discriminant analysis (17, 18) and cluster analysis were also investigated (19,20). Artificial neural network (ANN) is a branch of chemometrics that resolves regression or classification problems. The applications of ANN in separation science and chemistry have been reported widely (21-23). For pattern recognition analysis in clinical study, ANN was also proven to be a promising method (8). 

PLS (Partial Least Squares) regression was used for quantification and classification of aristeromycin and neplanocin A (Figure 4). Matlab was used for PCA (Principal Components Analysis) (according to the NIPALS algorithm) to identify correlations amongst the variables from the 882 wavenumbers and reduce the number of inputs for Discriminant Function Analysis (DFA) (first 15 PCA scores used) (Figure 5). 

The adaptive least squares (ALS) method [396, 585 — 588] is a modification of discriminant analysis which separates several activity classes e.g. data ordered by a rating score) by a single discriminant function. The method has been compared with ordinary regression analysis, linear discriminant analysis, and other multivariate statistical approaches in most cases the ALS approach was found to be superior to categorize any numbers of classes of ordered data. ORMUCS (ordered multicate-gorial classification using simplex technique) [589] is an ALS-related approach which... 

PCA is not only used as a method on its own but also as part of other mathematical techniques such as SIMCA classification (see section on parametric classification methods), principal component regression analysis (PCRA) and partial least-squares modelling with latent variables (PLS). Instead of original descriptor variables (x-variables), PCs extracted from a matrix of x-variables (descriptor matrix X) are used in PCRA and PLS as independent variables in a regression model. These PCs are called latent variables in this context. 

Romisch et al. in 2009 presented a study on the characterization and determination of the geographical origin of wines. In this paper, three methods of discrimination and classification of multivariate data were considered and tested the classification and regression trees (CART), the regularized discriminant analysis (RDA) and the partial least squares discriminant analysis (PLS-DA). PLS-DA analysis showed better classification results with percentage of correct classified samples from 88 to 100%. 

---