Multiple linear least squares regression

More on Multiple linear least squares regression (MLLSR), also known as Multiple linear regression (MLR) and P-matrix, and its sibling, K-matrix... 

OLS is synonymous with the following terms least squares regression, linear least squares regression, multiple least squares regression, multivariate least squares regression. OLS provides the best linear unbiased estimator (BLUE) that has the smallest variance among all linear and unbiased estimators. 

To gain insight into chemometric methods such as correlation analysis, Multiple Linear Regression Analysis, Principal Component Analysis, Principal Component Regression, and Partial Least Squares regression/Projection to Latent Structures... 

When experimental data is to be fit with a mathematical model, it is necessary to allow for the fact that the data has errors. The engineer is interested in finding the parameters in the model as well as the uncertainty in their determination. In the simplest case, the model is a linear equation with only two parameters, and they are found by a least-squares minimization of the errors in fitting the data. Multiple regression is just linear least squares applied with more terms. Nonlinear regression allows the parameters of the model to enter in a nonlinear fashion. See Press et al. (1986) for a description of maximum likelihood as it applies to both linear and nonlinear least squares. 

Chemometrics is the discipline concerned with the application of statistical and mathematical methods to chemical data [2.18], Multiple linear regression, partial least squares regression and the analysis of the main components are the methods that can be used to design or select optimal measurement procedures and experiments, or to provide maximum relevant chemical information from chemical data analysis. Common areas addressed by chemometrics include multivariate calibration, visualisation of data and pattern recognition. Biometrics is concerned with the application of statistical and mathematical methods to biological or biochemical data. 

NIR spectroscopy became much more useful when the principle of multiple-wavelength spectroscopy was combined with the deconvolution methods of factor and principal component analysis. In typical applications, partial least squares regression is used to model the relation between composition and the NIR spectra of an appropriately chosen series of calibration samples, and an optimal model is ultimately chosen by a procedure of cross-testing. The performance of the optimal model is then evaluated using the normal analytical performance parameters of accuracy, precision, and linearity. Since its inception, NIR spectroscopy has been viewed primarily as a technique of quantitative analysis and has found major use in the determination of water in many pharmaceutical materials. 

The common multivariate calibration methods are multiple linear regression, partial least-squares regression, and principal components regression. 

Near-infrared (NIR) spectroscopy is becoming an important technique for pharmaceutical analysis. This spectroscopy is simple and easy because no sample preparation is required and samples are not destroyed. In the pharmaceutical industry, NIR spectroscopy has been used to determine several pharmaceutical properties, and a growing literature exists in this area. A variety of chemoinfometric and statistical techniques have been used to extract pharmaceutical information from raw spectroscopic data. Calibration models generated by multiple linear regression (MLR) analysis, principal component analysis, and partial least squares regression analysis have been used to evaluate various parameters. 

Ordinary Least Square regression (OLS), also called Multiple Linear Regression (MLR), is the most common regression technique used to estimate the quantitative relationship between molecular descriptors and the property. Partial Least Squares (PLS) regression is widely applied especially when there are a large number of molecular descriptors with respect to the number of training compounds, as it happens for methods such as GRID and CoMFA. 

Quantitative Structure - Activity Relationships (QSARs) are estimation methods developed and used to predict certain effects or properties of chemical substances, which are primarily based on the structure of the chemicals. The development of QSARs often relies on the application of statistical methods such as multiple linear regression (MLR) or partial least squares regression (PLS). However, since toxicity data often include uncertainties and measurements errors, when the aim is to point out the more toxic and thus hazardous chemicals and to set priorities, order models can be used as alternative to statistical methods such as multiple linear regression. 

In the above examples there is a natural way to order the complete data set in two blocks, where both blocks have one mode in common. In Chapter 3 the methods of multiple linear regression, principal component regression, and partial least squares regression will be discussed on an introductory level. 

D. Broadhurst, R. Goodacre, A. Jones, J.J. Rowland and D.B. Kell, Genetic Algorithms as a Method for Variable Selection in Multiple Linear Regression and Partial Least Squares Regression, with Applications to Pyrolysis Mass Spectrometry, Analytica Chimica Acta, 348(1-3) (1997), 71-86. 

---