Latent variable regression

Partial Least Squares Regression (PLS) is a multivariate calibration technique, based on the principles of Latent Variable Regression. Originated in a slightly different form in the field of econometrics, PLS has entered the spectroscopic scene.46,47,48 It is mostly employed for quantitative analysis of mixtures with overlapping bands (e.g. mixture of glucose, fructose and sucrose).49,50... 

Trygg, J. and Wold, S., 02PLS, a two-block (x-y) latent variable regression (LVR) method with an integral OSC filter, J. Chemom., 17, 53, 2003. 

Kvalheim, O.M. Karstang, T.V. (1989). Interpretation of latent-variable regression models. Chemometrics and Intelligent Laboratory Systems, Vol. 7, No.1-2, (December 1989), pp. 39-51, ISSN 0169-7439... 

To conclude the list of approachesfor variable selection, we briefly mention the idea of using the output from an all variable selection run as input into a latent variables regression analysis." This technique is expensive computationally as it requires not only the subsets of variables to be found but also the number of latent variables needed to optimize some criterion function. 

Evaluation of the Benefits and Hazards of Variable Selection in Latent Variable Regression. Part I. Search Algorithms, Theory and Simulations. 

Figure 5 shows a typical plot of RMSE as a function of the number of factors from a latent variable regression model. 

Another problem is to determine the optimal number of descriptors for the objects (patterns), such as for the structure of the molecule. A widespread observation is that one has to keep the number of descriptors as low as 20 % of the number of the objects in the dataset. However, this is correct only in case of ordinary Multilinear Regression Analysis. Some more advanced methods, such as Projection of Latent Structures (or. Partial Least Squares, PLS), use so-called latent variables to achieve both modeling and predictions. 

Partial least-squares path modeling with latent variables (PLS), a newer, general method of handling regression problems, is finding wide apphcation in chemometrics. This method allows the relations between many blocks of data ie, data matrices, to be characterized (32—36). Linear and multiple regression techniques can be considered special cases of the PLS method. 

Other chemometrics methods to improve caUbration have been advanced. The method of partial least squares has been usehil in multicomponent cahbration (48—51). In this approach the concentrations are related to latent variables in the block of observed instmment responses. Thus PLS regression can solve the colinearity problem and provide all of the advantages discussed earlier. Principal components analysis coupled with multiple regression, often called Principal Component Regression (PCR), is another cahbration approach that has been compared and contrasted to PLS (52—54). Cahbration problems can also be approached using the Kalman filter as discussed (43). 

Partial least-squares in latent variables (PLS) is sometimes called partial least-squares regression, or PLSR. As we are about to see, PLS is a logical, easy to understand, variation of PCR. 

Partial least squares regression (PLS). Partial least squares regression applies to the simultaneous analysis of two sets of variables on the same objects. It allows for the modeling of inter- and intra-block relationships from an X-block and Y-block of variables in terms of a lower-dimensional table of latent variables [4]. The main purpose of regression is to build a predictive model enabling the prediction of wanted characteristics (y) from measured spectra (X). In matrix notation we have the linear model with regression coefficients b ... 

Additionally, Breiman et al. [23] developed a methodology known as classification and regression trees (CART), in which the data set is split repeatedly and a binary tree is grown. The way the tree is built, leads to the selection of boundaries parallel to certain variable axes. With highly correlated data, this is not necessarily the best solution and non-linear methods or methods based on latent variables have been proposed to perform the splitting. A combination between PLS (as a feature reduction method — see Sections 33.2.8 and 33.3) and CART was described by... 

We can go one step further, however. Each of the above multiple regression relations is between a single variable (response) of one data set and a linear combination of the variables (predictors) from the other set. Instead, one may consider the multiple-multiple correlation, i.e. the correlation of a linear combination from one set with a linear combination of the other set. Such linear combinations of the original variables are variously called factors, components, latent variables, canonical variables or canonical variates (also see Chapters 9,17, 29, and 31). 

A. Burnham, R. Viveros, J.F. MacGregor, Frameworks for latent variable multivariate regression. J. Chemom., 10 (1996) 31 6. 

On the other hand, when latent variables instead of the original variables are used in inverse calibration then powerful methods of multivariate calibration arise which are frequently used in multispecies analysis and single species analysis in multispecies systems. These so-called soft modeling methods are based, like the P-matrix, on the inverse calibration model by which the analytical values are regressed on the spectral data ... 

Multivariate calibration has the aim to develop mathematical models (latent variables) for an optimal prediction of a property y from the variables xi,..., jcm. Most used method in chemometrics is partial least squares regression, PLS (Section 4.7). An important application is for instance the development of quantitative structure—property/activity relationships (QSPR/QSAR). 

Regression can be performed directly with the values of the variables (ordinary least-squares regression, OLS) but in the most powerful methods, such as principal component regression (PCR) and partial least-squares regression (PLS), it is done via a small set of intermediate linear latent variables (the components). This approach has important advantages ... 

Matrix B consists of q loading vectors (of appropriate lengths), each defining a direction in the x-space for a linear latent variable which has maximum Pearson s correlation coefficient between y and jf for j = 1,..., q. Note that the regression coefficients for all y-variables can be computed at once by Equation 4.52, however,... 

A simple strategy for variable selection is based on the information of other multivariate methods like PCA (Chapter 3) or PLS regression (Section 4.7). These methods form new latent variables by using linear combinations of the regressor... 

PLS and PCR are linear methods (although nonlinear versions exist) and therefore the final latent variable that predicts the modeled property, y, is a linear combination of the original variables, just as in OLS (Equation 4.1). In general, the resulting regression coefficients are different when applying OLS, PCR, and PLS, and the prediction performances of the models are different. 

During model development, a relatively small number of PLS components (intermediate linear latent variables) are calculated which are internally used for regression. 

According to Equation 4.67, both vectors have to be normalized such that Xwi = IlFdll = 1. The scores to the found directions are the projections t1 =Xtv1 and U = Yc, and they are already normalized to length 1. The latent variable p is found by OLS regression according to the model (Equation 4.62) by... 

The remaining task is to robustly estimate the score vectors T that are needed in the above regression. According to the latent variable model (Equation 4.62) for the... 

---