Linear modeling using principal component regression

5 Linear modeling using principal component regression 

---

The kinds of calculations described above are done for all the molecules under investigation and then all the data (combinations of 3-point pharmacophores) are stored in an X-matrix of descriptors suitable to be submitted for statistical analysis. In theory, every kind of statistical analysis and regression tool could be applied, however in this study we decided to focus on the linear regression model using principal component analysis (PCA) and partial least squares (PLS) (Fig. 4.9). PCA and PLS actually work very well in all those cases in which there are data with strongly collinear, noisy and numerous X-variables (Fig. 4.9). 

Finally, the possibility to study with linear regression models using principal component analysis (PGA) and partial least squares (PLS) regression analysis pharmacophores as descriptors for the corresponding molecules represents an interesting and novel approach in QSAR. 

The usual objective of PC A is to reduce the dimensionality of a data matrix, or determine its intrinsic dimensionality. The PCs can also be used in other QSAR methods including linear regression models (termed principal component regression, PCR). However, PLS gives similar results and is generally preferred to PCR. 

To avoid over-fitting, a commonly used approach is to select a subset of descriptors to build models. GAs are widely used to select descriptors prior to using other statistical tools, such as MLR, to build models. Certainly, principal component analysis and PLS fitting are also widely used in reducing the dimensions of descriptors. Traditionally, stepwise linear regression is used to select certain descriptors to enter the regression equations. 

Principal component regression is typically used for linear regression models (Equation 6.7 or Equation 6.10), where the number of independent variables p is very large or where the regressors are highly correlated (this is known as multicollinearity). 

The prediction of Y-data of unknown samples is based on a regression method where the X-data are correlated to the Y-data. The multivariate methods, usually used for such a calibration, are principal component regression (PCR) and partial least squares regression (PLS). Both methods are based on the assumption of linearity and can deal with co-linear data. The problem of co-linearity is solved in the same way as the formation of a PCA plot. The X-variables are added together into latent variables, score vectors. These vectors are independent since they are orthogonal to each other and they can therefore be used to create a calibration model. 

For the estimation of components concentration, a second step is required, based on a multiple linear regression (MLR, see Section 3.1.3) between the absorbance values and the PCA scores. This can be carried out automatically after the PCA step, with the principal component regression (PCR) procedure (including PCA). This methodology was first applied to analytical chemical problems by Lawton and Sylvestre [25], and has more recently been used in different models by other researchers [26-28], Finally, the PCA procedure can also be coupled with cluster analysis (CA), as described in a very recent study on the characterisation of industrial wastewater samples [29],... 

In the arsenal of calibration methods there are methods more suited for modelling any number of correlated variables. The most popular among them are Principal Component Regression (PCR) and Partial Least Squares (PLS) [3], Their models are based on a few orthogonal latent variables, each of them being a linear combination of all original variables. As all the information contained in the spectra can be used for the modelling, these methods are often called the full-spectrum methods. ... 

The Principal Components Regression (PCR) method uses linear regression to generate a model by means of the prindpral compwnents as independent descriptors. PCR applies the scores from PCA as regressors in the QSAR model Therefore, a multiple-term linear equation is generated and derived from a paindjjal components analysis transformation of the independent variables. 

Quantitative analysis for one or more analytes through the simultaneous measurement of experimental parameters such as molecular UV or infrared absorbance at multiple wavelengths can be achieved even where clearly defined spectral bands are not discernible. Standards of known composition are used to compute and refine quantitative calibration data assuming linear or nonlinear models. Principal component regression (PCR) and partial least squares (PLS) regression are two multivariate regression techniques developed from linear regression (Topic B4) to optimize the data. 

PCR and PLS were developed to overcome the limitations of MLR. They use all the spectral data and so avoid the need for wavelength selection. PCR is essentially a mathematically more robust way of carrying out MLR. The regression is performed on the principal components of the data set. The principal components are determined from the data set with the specific aim that they will provide robust models. The principal components are linear combinations of the original measurements such that the first component explains the most variance in the data and subsequent components, all orthogonal, explain decreasing amounts of data variance. 

---