Partial least squares problem

Some methods that paitly cope with the above mentioned problem have been proposed in the literature. The subject has been treated in areas like Cheraometrics, Econometrics etc, giving rise for example to the methods Partial Least Squares, PLS, Ridge Regression, RR, and Principal Component Regression, PCR [2]. In this work we have chosen to illustrate the multivariable approach using PCR as our regression tool, mainly because it has a relatively easy interpretation. The basic idea of PCR is described below. 

Another problem is to determine the optimal number of descriptors for the objects (patterns), such as for the structure of the molecule. A widespread observation is that one has to keep the number of descriptors as low as 20 % of the number of the objects in the dataset. However, this is correct only in case of ordinary Multilinear Regression Analysis. Some more advanced methods, such as Projection of Latent Structures (or. Partial Least Squares, PLS), use so-called latent variables to achieve both modeling and predictions. 

The ability of partial least squares to cope with data sets containing very many x values is considered by its proponents to make it particularly suited to modern-day problems, where it is very easy to compute an extremely large number of descriptors for each compound (as in CoMFA). This contrasts with the traditional situation in QSAR, where it could be time-consuming to measure the required properties or where the analysis was restricted to traditional substituent constants. 

Partial least-squares path modeling with latent variables (PLS), a newer, general method of handling regression problems, is finding wide apphcation in chemometrics. This method allows the relations between many blocks of data ie, data matrices, to be characterized (32—36). Linear and multiple regression techniques can be considered special cases of the PLS method. 

Other chemometrics methods to improve caUbration have been advanced. The method of partial least squares has been usehil in multicomponent cahbration (48—51). In this approach the concentrations are related to latent variables in the block of observed instmment responses. Thus PLS regression can solve the colinearity problem and provide all of the advantages discussed earlier. Principal components analysis coupled with multiple regression, often called Principal Component Regression (PCR), is another cahbration approach that has been compared and contrasted to PLS (52—54). Cahbration problems can also be approached using the Kalman filter as discussed (43). 

S Wold, A Ruhe, H Wold, WJ Dunn III. The collmearity problem m linear regression. The partial least squares (PLS) approach to generalized inverses. SIAM I Sci Stat Comput 5 735-743, 1984. 

To (hopefully) help keep things simple, we will organize all of our data into column-wise matrices. Later on, when we explore Partial Least-Squares (PLS), we will have to remember that the PLS convention expects data to be organized row-wise. This isn t a great problem since one convention is merely the matrix transpose of the other. Nonetheless, it is one more thing we have to remember. 

L Stable and S. Wold, Partial least square analysis with cross-validation for the two-class problem a Monte Carlo study. J. Chemometrics, 1 (1987) 185-196. 

A difficulty with Hansch analysis is to decide which parameters and functions of parameters to include in the regression equation. This problem of selection of predictor variables has been discussed in Section 10.3.3. Another problem is due to the high correlations between groups of physicochemical parameters. This is the multicollinearity problem which leads to large variances in the coefficients of the regression equations and, hence, to unreliable predictions (see Section 10.5). It can be remedied by means of multivariate techniques such as principal components regression and partial least squares regression, applications of which are discussed below. 

The method of PCA can be used in QSAR as a preliminary step to Hansch analysis in order to determine the relevant parameters that must be entered into the equation. Principal components are by definition uncorrelated and, hence, do not pose the problem of multicollinearity. Instead of defining a Hansch model in terms of the original physicochemical parameters, it is often more appropriate to use principal components regression (PCR) which has been discussed in Section 35.6. An alternative approach is by means of partial least squares (PLS) regression, which will be more amply covered below (Section 37.4). 

B. Waiczack and D.L. Massart, Application of radial basis functions-partial least squares to non-linear pattern recognition problems diagnosis of process faults. Anal. Chim. Acta, 331 (1996) 187-193. 

In their day, Principal Components and Partial Least Squares were each considered almost as the magic answer to all calibration problems . It took a long time for the realization to dawn that they contain no magic and are subject to most of the... 

Wold, S Ruhe, A., Wold, H Dunn, W. J. I. SIAM J. Sci. Stat. Comput. 5, 1984, 735-743. The collinearity problem in linear regression. The partial least squares approach to generalized inverses. 

B program, PLS-2, uses the partial least squares (PLS) method. This method has been proposed by H. Wold (37) and was discussed by S. Wold (25). In such a problem there are two blocks of data, T and X. It is assumed that T is related to X by latent variables u and t is derived from the X block and u is derived from the Y block. 

The partial least squares (PLS) method has been applied to structure activity problems by Wold al. (38). Recently, Lindberg al. (40) employed this approach to resolve mixtures of humic acid and ligninsulfonate on the basic of fluorescence spectra. 

M. Sjostrom, S. Wold, W. Lindberg, J.A. Persson and H. Martens, A multivariate calibration problem in analytical chemistry solved by partial least squares models in latent variables. Anal. Chim. Acta, 150, 61-70 (1983). 

In the past few years, PLS, a multiblock, multivariate regression model solved by partial least squares found its application in various fields of chemistry (1-7). This method can be viewed as an extension and generalization of other commonly used multivariate statistical techniques, like regression solved by least squares and principal component analysis. PLS has several advantages over the ordinary least squares solution therefore, it becomes more and more popular in solving regression models in chemical problems. 

Haaland and coworkers (5) discussed other problems with classical least-squares (CLS) and its performance relative to partial least-squares (PLS) and factor analysis (in the form of principal component regression). One of the disadvantages of CLS is that interferences from overlapping spectra are not handled well, and all the components in a sample must be included for a good analysis. For a material such as coal LTA, this is a significant limitation. 

Problems like overlapping and interfering of fluorophores is overcome by the BioView sensor, which offers a comprehensive monitoring of the wide spectral range. Multivariate calibration models (e.g., partially least squares (PLS), principal component analysis (PCA), and neuronal networks) are used to filter information out of the huge data base, to combine different regions in the matrix, and to correlate different bioprocess variables with the courses of fluorescence intensities. 

This problem is overcome by the Bio View sensor, which offers the possibility to monitor the whole spectral range simultaneously, and by using suitable data analysis and mathematical methods like chemometric regression models 11061. Real-time fluorescence measurement can be used more effectively comparing time-consuming off-line methods. Partial least squares (PLS) calibration models were developed for simultaneous on-line prediction of the cell dry mass concentration (Fig. 5), product concentration (Fig. 6), and metabolite concentrations (e. g., acetic acid, not shown) from 2D spectra. 

J. Amador-Hernandez, L. E. Garcia-Ayuso, J. M. Fernandez-Romero and M. D. Luque de Castro, Partial least squares regression for problem solving in precious metal analysis by laser induced breakdown spectrometry, J. Anal. At. Spectrom., 15, 2000, 587-593. 

---