Multivariate least squares analysis

In multivariate least squares analysis, the dependent variable is a function of two or more independent variables. Because matrices are so conveniently handled by computer and because the mathematical formalism is simpler, multivariate analysis will be developed as a topic in matrix algebra rather than conventional algebra. 

Haaland, D.M., et. al. "Multivariate Least-Squares Methods Applied to the Quantitative Spectral Analysis of Multicomponent Samples", Appl. Spec. 1985 (39) 73-84. 

Reduced rank regression (RRR), also known as redundancy analysis (or PCA on Instrumental Variables), is the combination of multivariate least squares regression and dimension reduction [7]. The idea is that more often than not the dependent K-variables will be correlated. A principal component analysis of Y might indicate that A (A m) PCs may explain Y adequately. Thus, a full set of m... 

This method measures total unsaturation in a sample by the multivariate analysis of Fourier transform infrared spectra. It correlates the absorbance in the spectral regions corresponding to two major types of unsaturation with their concentrations. This is an extension of univariate least squares analysis that correlates a single band absorbance height or area with concentration. 

To determine the oil, water, and solids contents simultaneously, sophisticated statistical techniques must usually be applied, such as partial least-squares analysis (PLS) and multivariate analysis (MVA). This approach requires a great deal of preparation and analysis of standards for calibration. Near-infrared peaks can generally be quantified by using Beer s law consequently, NIRA is an excellent analytical tool. In addition, NIRA has a fast spectral acquisition time and can be adapted to fiber optics this adaptability allows the instrument to be placed in a control room somewhat isolated from the plant environment. 

A different approach to mathematical analysis of the solid-state C NMR spectra of celluloses was introduced by the group at the Swedish Forest Products Laboratory (STFI). They took advantage of statistical multivariate data analysis techniques that had been adapted for use with spectroscopic methods. Principal component analyses (PCA) were used to derive a suitable set of subspectra from the CP/MAS spectra of a set of well-characterized cellulosic samples. The relative amounts of the I and I/3 forms and the crystallinity index for these well-characterized samples were defined in terms of the integrals of specific features in the spectra. These were then used to derive the subspectra of the principal components, which in turn were used as the basis for a partial least squares analysis of the experimental spectra. Once the subspectra of the principal components are validated by relating their features to the known measures of variability, they become the basis for analysis of the spectra of other cellulosic samples that were not included in the initial analysis. Here again the interested reader can refer to the original publications or the overview presented earlier. ... 

For overlapping peaks the data matrix contains linear combinations of the pure spectra of the overlapping components in its rows, and combinations of the pure elution profiles in its columns. Multivariate analysis of the data matrix may allow extraction of useful information from either the rows or columns of the matrix, or an edited form of the data matrix [107,116-118]. Factor analysis approaches or partial least-squares analysis can provide information on whether a given spectrum (known compound) or several known compounds are present in a peak. Principal component analysis and factor analysis can be used to estimate the maximum number of probable (unknown) components in a peak cluster. Deconvolution or iterative target factor analysis can then be used to estimate the relative concentration of each component with known spectra in a peak cluster. 

In this illustration, we have only considered one dependent (y) variable, LD25. In fact, partial least squares can deal with multivariate problems, where there is more than one dependent variable, such as different measures of biological activity or different properties. Indeed, for the above set of compounds five different measures of biological activity were reported in the original paper and a partial least-squares analysis performed on the entire data set [Dunn et al. 1984]. The algorithm effectively finds pairs of vectors through both the x data and the y data such that the vector pairs are maximally correlated with each other whilst simultaneously explaining as much of the variance in their individual data blocks as possible... 

Principal component analysis and partial least squares analysis are chemometric tools for extracting and rationalizing the information from any multivariate description of a biological system. Complexity reduction and data simplification are two of the most important features of such tools. PCA and PLS condense the overall information into two smaller matrices, namely the score plot (which shows the pattern of compounds) and the loading plot (which shows the pattern of descriptors). Because the chemical interpretation of score and loading plots is simple and straightforward, PCA and PLS are usually preferred to other nonlinear methods, especially when the noise is relatively high. ... 

In this section we shall consider the rather general case where for a series of chemical compounds measurements are made in a number of parallel biological tests and where a set of descriptor variables is believed to be related to the biological potencies observed. In order to imderstand the data in their entirety and to deal adequately with the mathematical properties of such data, methods of multivariate statistics are required. A variety of such methods is available as, for example, multivariate regression, canonical correlation, principal component analysis, principal component regression, partial least squares analysis, and factor analysis, which have all been applied to biological or chemical problems (for reviews, see [1-11]). Which method to choose depends on the ultimate objective of an analysis and the property of the data. We have found principal component and factor analysis particularly useful. For this reason and also since many multivariate methods make use of components for factors we will start with these methods in some detail, while the discussion of other approaches will be less extensive. 

Haaland DM, Easterling RG, Vopicka DA. Multivariate least-squares methods applied to the quantitative spectral analysis of multicomponent samples. Appl Spectrosc 1985 39 73-84. 

Sections 9A.2-9A.6 introduce different multivariate data analysis methods, including Multiple Linear Regression (MLR), Principal Component Analysis (PCA), Principal Component Regression (PCR) and Partial Least Squares regression (PLS). 

The total residual sum of squares, taken over all elements of E, achieves its minimum when each column Cj separately has minimum sum of squares. The latter occurs if each (univariate) column of Y is fitted by X in the least-squares way. Consequently, the least-squares minimization of E is obtained if each separate dependent variable is fitted by multiple regression on X. In other words the multivariate regression analysis is essentially identical to a set of univariate regressions. Thus, from a methodological point of view nothing new is added and we may refer to Chapter 10 for a more thorough discussion of theory and application of multiple regression. 

An alternative and illuminating explanation of reduced rank regression is through a principal component analysis of Y, the set of fitted F-variables resulting from an unrestricted multivariate multiple regression. This interpretation reveals the two least-squares approximations involved projection (regression) of Y onto X, followed by a further projection (PCA) onto a lower dimensional subspace. 

A difficulty with Hansch analysis is to decide which parameters and functions of parameters to include in the regression equation. This problem of selection of predictor variables has been discussed in Section 10.3.3. Another problem is due to the high correlations between groups of physicochemical parameters. This is the multicollinearity problem which leads to large variances in the coefficients of the regression equations and, hence, to unreliable predictions (see Section 10.5). It can be remedied by means of multivariate techniques such as principal components regression and partial least squares regression, applications of which are discussed below. 

Manne R (1987) Analysis of two partial-least-squares algorithms for multivariate calibration. Chemom Intell Lab Syst 2 187... 

Partial least squares (PLS) projections to latent structures [40] is a multivariate data analysis tool that has gained much attention during past decade, especially after introduction of the 3D-QSAR method CoMFA [41]. PLS is a projection technique that uses latent variables (linear combinations of the original variables) to construct multidimensional projections while focusing on explaining as much as possible of the information in the dependent variable (in this case intestinal absorption) and not among the descriptors used to describe the compounds under investigation (the independent variables). PLS differs from MLR in a number of ways (apart from point 1 in Section 16.5.1) ... 

A total of 185 emission lines for both major and trace elements were attributed from each LIBS broadband spectrum. Then background-corrected, summed, and normalized intensities were calculated for 18 selected emission lines and 153 emission line ratios were generated. Finally, the summed intensities and ratios were used as input variables to multivariate statistical chemometric models. A total of 3100 spectra were used to generate Partial Least Squares Discriminant Analysis (PLS-DA) models and test sets. 

The four-volume Handbook of Chemoinformatics—From Data to Knowledge (Gasteiger 2003) contains a number of introductions and reviews that are relevant to chemometrics Partial Least Squares (PLS) in Cheminformatics (Eriksson et al. 2003), Inductive Learning Methods (Rose 1998), Evolutionary Algorithms and their Applications (von Homeyer 2003), Multivariate Data Analysis in Chemistry (Varmuza 2003), and Neural Networks (Zupan 2003). 

---