The PLS Algorithm

The procedure for calculating the PLS model is given by several authors, amongst others, Geladi and Kowalski (1986). The procedure starts with assuming a score vector told that is any column of matrix X, for example Xi, and a score vector u which is any column of matrix T, for example j i. Assuming that X and Y are auto-scaled, the following steps are then carried out  

---

PLS has been introduced in the chemometrics literature as an algorithm with the claim that it finds simultaneously important and related components of X and of Y. Hence the alternative explanation of the acronym PLS Projection to Latent Structure. The PLS factors can loosely be seen as modified principal components. The deviation from the PCA factors is needed to improve the correlation at the cost of some decrease in the variance of the factors. The PLS algorithm effectively mixes two PCA computations, one for X and one for Y, using the NIPALS algorithm. It is assumed that X and Y have been column-centred as usual. The basic NIPALS algorithm can best be demonstrated as an easy way to calculate the singular vectors of a matrix, viz. via the simple iterative sequence (see Section 31.4.1) ... 

The PLS algorithm is relatively fast because it only involves simple matrix multiplications. Eigenvalue/eigenvector analysis or matrix inversions are not needed. The determination of how many factors to take is a major decision. Just as for the other methods the right number of components can be determined by assessing the predictive ability of models of increasing dimensionality. This is more fully discussed in Section 36.5 on validation. 

In principle, in the absence of noise, the PLS factor should completely reject the nonlinear data by rotating the first factor into orthogonality with the dimensions of the x-data space which are spawned by the nonlinearity. The PLS algorithm is supposed to find the (first) factor which maximizes the linear relationship between the x-block scores and the y-block scores. So clearly, in the absence of noise, a good implementation of PLS should completely reject all of the nonlinearity and return a factor which is exactly linearly related to the y-block variances. (Richard Kramer)... 

Some of this variance was indeed rejected by the PLS algorithm, but the amount, compared to the Principal Component algorithm, seems to have been rather minuscule, rather than providing a nearly exact fit. 

Traditional macroscale NIR spectroscopy requires a calibration set, made of the same chemical components as the target sample, but with varying concentrations that are chosen to span the range of concentrations possible in the sample. A concentration matrix is made from the known concentrations of each component. The PLS algorithm is used to create a model that best describes the mathematical relationship between the reference sample data and the concentration matrix. The model is applied to the unknown data from the target sample to estimate the concentration of sample components. This is called concentration mode PLS . 

These same analysis techniques can be applied to chemical imaging data. Additionally, because of the huge number of spectra contained within a chemical imaging data set, and the power of statistical sampling, the PLS algorithm can also be applied in what is called classification mode as described in Section 8.4.5. When the model is applied to data from the sample, each spectrum is scored relative to its membership to a particular class (i.e. degree of purity relative to a chemical component). Higher scores indicate more similarity to the pure component spectra. While these scores are not indicative of the absolute concentration of a chemical component, the relative abundance between the components is maintained, and can be calculated. If all sample components are accounted for, the scores for each component can be normalized to unity, and a statistical assessment of the relative abundance of the components made. 

The potentials of the PLS algorithm are very well demonstrated on the spectro-fluorimetric analysis of mixtures of hiunic acid and Ugninsulfonate investigated by Lindberg et al. The problems associated with this analysis are the strong similarities... 

Once the basis of the PLS algorithm has been presented, it is easier to understand the advantages of this multivariate regression technique over the simpler ones MLR and PCR. Some of these advantages are already obtained in PCR. They can be summarised as follows ... 

These same analysis techniques can be applied to chemical imaging data. Additionally, because of the huge number of spectra contained within a chemical imaging data set, and the power of statistical sampling, the PLS algorithm can also be applied in what is called classification mode. In this case, the reference library used to establish the PLS model is... 

In order to handle multiple Y-variables, an extension of the PLS regression method discussed earlier, called PLS-2, must be used.1 The algorithm for the PLS-2 method is quite similar to the PLS algorithms discussed earlier. Just like the PLS method, this method determines each compressed variable (latent variable) based on the maximum variance explained in both X and Y. The only difference is that Y is now a matrix that contains several Y-variables. For PLS-2, the second equation in the PLS model (Equation 8.36) can be replaced with the following ... 

The advance of the PLS method is the nonproblematic handling of multicollinearities. In contrast with the other methods of multivariate data analysis the PLS algorithm is an iterative algorithm which makes it possible to treat data which have more features than objects [GELADI, 1988],... 

Various descriptions of the PLS algorithm exist in the literature. Some of the differences arise from the way normalization is used. In some descriptions, neither the scores nor the loadings are normalized. In other descriptions, either the loadings or scores may be normalized. These differences result in different expressions for the PLS calculations however, the estimated regression vectors for b should be the same, except for differences in round-off error. 

Several steps are involved in rapid analysis method development. These include gathering appropriate calibration samples, chemical characterization of the calibration samples, developing spectroscopic methods for the rapid technique, projection-to-latent-structures (PLS) regression, validation of the PLS algorithm, and the development of QA/QC procedures.128... 

It is, however, not necessary to compute eigenvectors by diagonalization of matrices. The PLS algorithm is based upon the NIPALS algorithm which makes it possible to iteratively determine one PLS dimension at a time. For details of the PLS algorithm, see [75]. 

Step 3. For every random combination of descriptors (i.e., every parent), a QSAR equation is generated for the training data set by use of the PLS algorithm (41). Thus, for each parent a value is obtained, and some function of is used as a fitness function to guide GA. 

Ge and ZnSe IREs were systematic, could be modeled to some extent by the PLS algorithm, and did not result from protein interactions with the IREs. 

Several enhancements have been made to the PLS algorithm [48, 93, 169, 184, 336, 333, 339]. Commercial software is available for developing PLS models [328, 269]. 

Note that the algorithms in Equations (3.45) and (3.46) are noniterative, because the y is univariate. For multivariate Y, the PLS algorithms become iterative. There are alternative algorithms for partial least squares regression, depending on the size of X [De Jong Ter Braak 1994, Lindgren et al. 1993], Moreover, it is a matter of choice whether to deflate X and/or y [Burnham et al. 1996],... 

The PLS algorithm then minimizes F while preserving the correlation between X and y through the equation U = B x T. 

Partial Least Squares (PLS) is an extension of PCA where both the x and y data are considered. In PCA only the x data is considered. The goal of the PLS analysis is to build an equation that predicts y values (laboratory data) based on x (spectral) data. The PLS equation or calibration is based on decomposing both the x and y data into a set of scores and loadings, similar to PCA. However, the scores for both the X and y data are not selected based on the direction of maximum variation but are selected in order to maximize the correlation between the scores for both the x and y variables. As with PCA, in the PLS regression development the number of components or factors is an important practical consideration. A more detailed discussion of the PLS algorithm can be found elsewhere [13, 14]. Commercial software can be used to construct and optimize both PCA and PLS calibration models. 

One can show that the matrix P W is an upper bidiagonal matrix so that the PLS algorithm represents just a variation of diagonalizing a matrix before its inversion. 

---