Regression on principal components and partial least squares

Another aspect of the dimensionality of a data set that is perhaps not quite so obvious concerns the number of members in each class. Ideally, each of the classes in a data set should contain about the same number of members. If one class contains only a small number of samples, say ten per cent of the total points or less, then the discriminant function may be able to achieve a trivial separation despite the fact that the ratio of points to descriptors for the overall data set is greater than three. The following guidelines should be borne in mind when applying discriminant techniques. 

Finally, it may be the case that the data is not capable of linear separation. Such a situation is shown in Fig. 7.5 where one class is embedded within the other. A recently described technique for the treatment of such data sets makes use of PCs scaled according to the parameter values of the class of most interest, usually the actives (Rose et al. 1991). This is somewhat reminiscent of the SIMCA method. 

3 Regression on principal components and partial least squares 

Methods such as PCA (see Section 4.2) and factor analysis (FA) (see Section S.3) are data-reduction techniques which result in the creation of new variables from linear combinations of the original variables. These new variables have an important quality, orthogonality, which makes them particularly suitable for use in the construction of regression models. They are also sorted in order of importance, in so far as the amount of variance 

---

A general requirement for P-matrix analysis is n = rank(R). Unfortcmately, for most practical cases, the rank of R is greater than the number of components, i.e., rank(R) > n, and rank(R) = min(m, p). Thus, P-matrix analysis is associated with the problem of substituting R with an R that produces rank(R ) = n. This is mostly done by orthogonal decomposition methods, such as principal components analysis, partial least squares (PLS), or continuum regression [4]. Dimension requirements of involved matrices for these methods are m > n, and p > n. If the method of least squares is used, additional constraints on matrix dimensions are needed [4]. The approach of P-matrix analysis does not require quantitative concentration information of all constituents. Specifically, calibration samples with known concentrations of analytes under investigation satisfy the calibration needs. The method of PLS will be used in this chapter for P-matrix analysis. 

This chapter ends with a short description of the important methods, Principal Component Regression (PCR) and Partial Least-Squares (PLS). Attention is drawn to the similarity of the two methods. Both methods aim at predicting properties of samples based on spectroscopic information. The required information is extracted from a calibration set of samples with known spectrum and property. 

Principal Component Regression, PCR, and Partial Least Squares, PLS, are the most widely known and applied chemometrics methods. This is particularly the case for PLS, for which there is a tremendous number of applications and a never-ending stream of proposed improvements. The details of these latest modifications are not within the scope of this book and we concentrate on the essential, classical aspects. 

The calibration methods most frequently used to relate the property to be measured to the analytical signals acquired in NIR spectroscopy are MLR,59 60 principal component regression (PCR)61 and partial least-squares regression (PLSR).61 Most of the earliest quantitative applications of NIR spectroscopy were based on MLR because spectra were then recorded on filter instruments, which afforded measurements at a relatively small number of discrete wavelengths only. However, applications involving PCR and PLSR... 

Although the emphasis in this chapter is on multiple hnear regression techniques, it is important to recognise that the analysis of design experiments is not restricted to such approaches, and it is legitimate to employ multivariate methods such as principal components regression and partial least squares as described in detail in Chapter 5. 

The kinds of calculations described above are done for all the molecules under investigation and then all the data (combinations of 3-point pharmacophores) are stored in an X-matrix of descriptors suitable to be submitted for statistical analysis. In theory, every kind of statistical analysis and regression tool could be applied, however in this study we decided to focus on the linear regression model using principal component analysis (PCA) and partial least squares (PLS) (Fig. 4.9). PCA and PLS actually work very well in all those cases in which there are data with strongly collinear, noisy and numerous X-variables (Fig. 4.9). 

Linear discriminant analysis (LDA) is aimed at finding a linear combination of descriptors that best separate two or more classes of objects [100]. The resulting transformation (combination) may be used as a classifier to separate the classes. LDA is closely related to principal component analysis and partial least square discriminant analysis (PLS-DA) in that all three methods are aimed at identifying linear combinations of variables that best explain the data under investigation. However, LDA and PLS-DA, on one hand, explicitly attempt to model the difference between the classes of data whereas PCA, on the other hand, tries to extract common information for the problem at hand. The difference between LDA and PLS-DA is that LDA is a linear regression-like method whereas PLS-DA is a projection technique... 

The prediction of Y-data of unknown samples is based on a regression method where the X-data are correlated to the Y-data. The multivariate methods, usually used for such a calibration, are principal component regression (PCR) and partial least squares regression (PLS). Both methods are based on the assumption of linearity and can deal with co-linear data. The problem of co-linearity is solved in the same way as the formation of a PCA plot. The X-variables are added together into latent variables, score vectors. These vectors are independent since they are orthogonal to each other and they can therefore be used to create a calibration model. 

In the above examples there is a natural way to order the complete data set in two blocks, where both blocks have one mode in common. In Chapter 3 the methods of multiple linear regression, principal component regression, and partial least squares regression will be discussed on an introductory level. 

In the arsenal of calibration methods there are methods more suited for modelling any number of correlated variables. The most popular among them are Principal Component Regression (PCR) and Partial Least Squares (PLS) [3], Their models are based on a few orthogonal latent variables, each of them being a linear combination of all original variables. As all the information contained in the spectra can be used for the modelling, these methods are often called the full-spectrum methods. ... 

PCA is not only used as a method on its own but also as part of other mathematical techniques such as SIMCA classification (see section on parametric classification methods), principal component regression analysis (PCRA) and partial least-squares modelling with latent variables (PLS). Instead of original descriptor variables (x-variables), PCs extracted from a matrix of x-variables (descriptor matrix X) are used in PCRA and PLS as independent variables in a regression model. These PCs are called latent variables in this context. 

On the other hand, atomic emission spectra are inherently well suited for multivariate analysis due to the fact that the intensity data can be easily recorded at multiple wavelengths. The only prerequisite is that the cahbration set encompasses all likely constituents encountered in the real sample matrix. Calibration data are therefore acquired by a suitable experimental design. Not surprisingly, many of the present analytical schemes are based on multivariate calibration techniques such as multiple linear regression (MLR), principal components regression (PCR), and partial least squares regression (PLS), which have emerged as attractive alternatives. 

Studies on binary mixture samples frequently deal with classical least-squares, inverse least-squares, principal component regression and partial least-squares methods. These methods have been used for resolving mixtures of hydrochlorothiazide and spironolactone in tablets cyproterone acetate and estradiol valerate amiloride and hydrochlorothiazide ... 

Leonard and Roy [ 194] recently reported QSAR 70-73 on the HIV protease inhibitory data of 1,2,5,6-tetra-o-benzyl-D-mannitols (62) studied by Bouzide et al. [195]. Several statistical techniques such as stepwise regression, multiple linear regression with factor analysis as the data preprocessing step (FA-MLR), principal component regression analysis (PCRA) and partial least square (PLS) analysis were appHed to identify the structural and physicochemical requirements for HIV protease inhibitory activity. 

The effect of sugar and acid contents in fruit on the transmitted output power was fully examined and used to estimate the optical parameter preferable for detecting internal quality. The performances of multiple Unear regression (MLR), principal component regression (PCR), and partial least-squares regression (PLS) analysis by TOF-NIRS were also compared to those from normal methods using reflectance data. 

---