Principal component regression,

This is a calibration technique (i.e supervised) that yields quantitative information about the components in the sample according to the relationship 

Note that this equation describes the relationship between concentration C of the component and the sensor response X. It is purposely written backwards by comparison with the usual notation used with linear sensors (e.g., optical, amperometric, etc.) discussed earlier. This convention helps to define P as the matrix of regression coefficients. 

For a well-behaved sensor array, only a small subset k of n available PCs is sufficient to characterize the matrix. Once again, Principal Component Regression (PCR) is a data reduction tool. The robustness of the selection of k can be tested by cross-validation in which case data subsets are randomly selected and the error matrix H xn is calculated. 

PCA has given us a way whereby we can take a calibration set of full absorbance spectra and decompose them into a small number of principal components. The principal components are needed to find the scores of the principal components in unknown spectra, and the scores are used in place of the absorbance spectra in either a CLS or an ILS model. The next two sections present two methods that make use of the principal components and their scores. 

Principal component regression (PCR) is the algorithm by which PCA is used for quantitative analysis and involves a two-step process. The first is to decompose a calibration data set with PCA to calculate all the significant principal components, and the second step is to regress the concentrations against the scores to produce the component calibration coefficients. Generally, the ILS model is preferred, as it does not require knowledge of the complete composition of all the spectra. Therefore, if we use the ILS model from Eq. 9.16 but rewrite it for scores, S, instead of absorbances. A, we have 

A characteristic of the principal component matrix is that it is an orthogonal matrix. When it is multiplied by its own transpose, the identity matrix, I, is the result. An identity matrix when multiplied with any other matrix results in the other matrix. I does not alter the other matrix in any way. Therefore, if we multiply Eq. 9.20 by the transpose of F, F, we have 

Now that we have S, we can proceed with the calculation of X from Eq. 9.21  

To measure the concentrations of components in an unknown mixture, we must first calculate the scores for the unknown mixture spectrum. From Eq. 9.22, 

Recall that, in order to generate an ILS calibration, we must have at least as many samples as there are wavelengths used in the calibration. Since we only have 15 spectra in our training sets but each spectrum contains 100 wavelengths, we were forced to find a way to reduce the dimensionality of our spectra to 15 or less. We have seen that principal component analysis (PCA) provides us with a way of optimally reducing the dimensionality of our data without degrading it, and with the added benefit of removing some noise. 

Even though we have waited until this point to discuss optional pretreatments, they are equally applicable to CLS, ILS, PCR, and PLS. There are a number of possible ways to pretreat our data before we find the principal components and perform the regression. They fall into 3 main categories  

Optional pretreatments can be applied, in any combination, to either the spectra (the x-data), the concentrations (the y-data) or both. 

Artifact removal and/or linearization. A common form of artifact removal is baseline correction of a spectrum or chromatogram. Common linearizations are the conversion of spectral transmittance into spectral absorbance and the multiplicative scatter correction for diffuse reflectance spectra. We must be very careful when attempting to remove artifacts. If we do not remove them correctly, we can actually introduce other artifacts that are worse than the ones we are trying to remove. But, for every artifact that we can correctly remove from the data, we make available additional degrees-of-freedom that the model can use to fit the relationship between the concentrations and the absorbances. This translates into greater precision and robustness of the calibration. Thus, if we can do it properly, it is always better to remove an artifact than to rely on the calibration to fit it. Similar reasoning applies to data linearization. 

Whether or not we scale, weight, and/or center our data, a mandatory pretreatment is required by most of the algorithms used to calculate the eigenvectors. Most algorithms require that we square our data matrix, A, by either pre- or post-multiplying it by its transpose  

---

Some methods that paitly cope with the above mentioned problem have been proposed in the literature. The subject has been treated in areas like Cheraometrics, Econometrics etc, giving rise for example to the methods Partial Least Squares, PLS, Ridge Regression, RR, and Principal Component Regression, PCR [2]. In this work we have chosen to illustrate the multivariable approach using PCR as our regression tool, mainly because it has a relatively easy interpretation. The basic idea of PCR is described below. 

It may look weird to treat the Singular Value Decomposition SVD technique as a tool for data transformation, simply because SVD is the same as PCA. However, if we recall how PCR (Principal Component Regression) works, then we are really allowed to handle SVD in the way mentioned above. Indeed, what we do with PCR is, first of all, to transform the initial data matrix X in the way described by Eqs. (10) and (11). 

To gain insight into chemometric methods such as correlation analysis, Multiple Linear Regression Analysis, Principal Component Analysis, Principal Component Regression, and Partial Least Squares regression/Projection to Latent Structures... 

Other chemometrics methods to improve caUbration have been advanced. The method of partial least squares has been usehil in multicomponent cahbration (48—51). In this approach the concentrations are related to latent variables in the block of observed instmment responses. Thus PLS regression can solve the colinearity problem and provide all of the advantages discussed earlier. Principal components analysis coupled with multiple regression, often called Principal Component Regression (PCR), is another cahbration approach that has been compared and contrasted to PLS (52—54). Cahbration problems can also be approached using the Kalman filter as discussed (43). 

Factor spaces are a mystery no more We now understand that eigenvectors simply provide us with an optimal way to reduce the dimensionality of our spectra without degrading them. We ve seen that, in the process, our data are unchanged except for the beneficial removal of some noise. Now, we are ready to use this technique on our realistic simulated data. PCA will serve as a pre-processing step prior to ILS. The combination of Principal Component Analysis with ILS is called Principal Component Regression, or PCR. 

As we saw in the last chapter, by discarding the noise eigenvectors, we are able to remove a portion of the noise from our data. We have called the data that results after the noise removal the regenerated data. When we perform principal component regression, there is not really a separate, explicit data regeneration step. By operating with the new coordinate system, we are automatically regenerating the data without the noise. 

Because of peak overlappings in the first- and second-derivative spectra, conventional spectrophotometry cannot be applied satisfactorily for quantitative analysis, and the interpretation cannot be resolved by the zero-crossing technique. A chemometric approach improves precision and predictability, e.g., by the application of classical least sqnares (CLS), principal component regression (PCR), partial least squares (PLS), and iterative target transformation factor analysis (ITTFA), appropriate interpretations were found from the direct and first- and second-derivative absorption spectra. When five colorant combinations of sixteen mixtures of colorants from commercial food products were evaluated, the results were compared by the application of different chemometric approaches. The ITTFA analysis offered better precision than CLS, PCR, and PLS, and calibrations based on first-derivative data provided some advantages for all four methods. ... 

Section 35.4), reduced rank regression (Section 35.5), principal components regression (Section 35.6), partial least squares regression (Section 35.7) and continuum regression methods (Section 35.8). 

The computational implementation of principal components regression is very straightforward. 

We have seen that PLS regression (covariance criterion) forms a compromise between ordinary least squares regression (OLS, correlation criterion) and principal components regression (variance criterion). This has inspired Stone and Brooks [15] to devise a method in such a way that a continuum of models can be generated embracing OLS, PLS and PCR. To this end the PLS covariance criterion, cov(t,y) = s, s. r, is modified into a criterion T = r. (For... 

Y.L. Xie and J.H. Kalivas, Evaluation of principal component selection methods to form a global prediction model by principal component regression. Anal. Chim. Acta (1997) 348, 19-27. 

M. Stone and R.J. Brooks, Continuum regression cross-validated sequentially constructed prediction embracing ordinary least sqaures, partial least squares, and principal component regression. J. Roy. Stat. Soc. B52 (1990) 237-269. 

We will see that CLS and ILS calibration modelling have limited applicability, especially when dealing with complex situations, such as highly correlated predictors (spectra), presence of chemical or physical interferents (uncontrolled and undesired covariates that affect the measurements), less samples than variables, etc. More recently, methods such as principal components regression (PCR, Section 17.8) and partial least squares regression (PLS, Section 35.7) have been... 

The application of principal components regression (PCR) to multivariate calibration introduces a new element, viz. data compression through the construction of a small set of new orthogonal components or factors. Henceforth, we will mainly use the term factor rather than component in order to avoid confusion with the chemical components of a mixture. The factors play an intermediary role as regressors in the calibration process. In PCR the factors are obtained as the principal components (PCs) from a principal component analysis (PC A) of the predictor data, i.e. the calibration spectra S (nxp). In Chapters 17 and 31 we saw that any data matrix can be decomposed ( factored ) into a product of (object) score vectors T(nxr) and (variable) loadings P(pxr). The number of columns in T and P is equal to the rank r of the matrix S, usually the smaller of n or p. It is customary and advisable to do this factoring on the data after columncentering. This allows one to write the mean-centered spectra Sq as ... 

The suffix in T (nxA) and P (< xA) indicates that only the first A columns of T and P are used, A being much smaller than n and q. In principal component regression we use the PC scores as regressors for the concentrations. Thus, we apply inverse calibration of the property of interest on the selected set of factor scores ... 

J.M. Sutter, J.H. Kalivas and P.M. Lang, Which principal components to utilize for principal component regression. J. Chemometr., 6 (1992) 217-225. 

E. Vigneau, D. Bertrand and E.M. Qannari, Application of latent root regression for calibration in near-infrared spectroscopy. Comparison with principal component regression and partial least squares. Chemometr. Intell. Lab. Syst., 35 (1996) 231-238. 

A difficulty with Hansch analysis is to decide which parameters and functions of parameters to include in the regression equation. This problem of selection of predictor variables has been discussed in Section 10.3.3. Another problem is due to the high correlations between groups of physicochemical parameters. This is the multicollinearity problem which leads to large variances in the coefficients of the regression equations and, hence, to unreliable predictions (see Section 10.5). It can be remedied by means of multivariate techniques such as principal components regression and partial least squares regression, applications of which are discussed below. 

The method of PCA can be used in QSAR as a preliminary step to Hansch analysis in order to determine the relevant parameters that must be entered into the equation. Principal components are by definition uncorrelated and, hence, do not pose the problem of multicollinearity. Instead of defining a Hansch model in terms of the original physicochemical parameters, it is often more appropriate to use principal components regression (PCR) which has been discussed in Section 35.6. An alternative approach is by means of partial least squares (PLS) regression, which will be more amply covered below (Section 37.4). 

---