The Multiple Linear Regression Model

This section introduces the regression theory that is needed for the establishment of the calibration models in the forthcoming sections and chapters. The multivariate linear models considered in this chapter relate several independent variables (x) to one dependent variable (y) in the form of a first-order polynomial  

In order to estimate the coefficients, / sets of observations on Xj,. . . , Xj and y are collected. These observations are obtained from a set of / calibration standards, for which the concentrations and spectra are known. The application of eqn (3.5) to these I standards yields a system of equations  

Ordinary Multiple Linear Regression and Principal Components Regression 

---

This problem corresponds to the multiple linear regression model with m= I, n=2 and p=3. In this case we take X =x2, x2=x and with Q,=l (all data points are weighed equally) Equations 3.19a and 3.19b become... 

More specifically, let us consider the multiple linear regression model y = Xb + e, see Equation 4.36, which can be denoted for each object as... 

The multiple linear regression model is simply an extension of the linear regression model (Equation 12.7), and is given below ... 

MLR is an inverse method that uses the multiple linear regression model that was discussed earlier [1,46] ... 

Like MLR, PCR [63] is an inverse calibration method. However, in PCR, the compressed variables (or PCs) from PCA are used as variables in the multiple linear regression model, rather than selected original X variables. In PCR, PCA is first done on the calibration x data, thus generating PCA scores (T) and loadings (P) (see Section 12.2.5), then a multiple linear regression is carried out according to the following model ... 

The multiple linear regression model assumes that in addition to the p independent x-variables, a response variable y is measured, which can be explained as an affine combination of the x-variables (also called the regressors). More precisely, the model says that for all observations (x, , y, ) with i = 1,. .., , it holds that... 

The multiple linear regression models are validated using standard statistical techniques. These techniques include inspection of residual plots, standard deviation, and multiple correlation coefficient. Both regression and computational neural network models are validated using external prediction. The prediction set is not used for descriptor selection, descriptor reduction, or model development, and it therefore represents a true unknown data set. In order to ascertain the predictive power of a model the rms error is computed for the prediction set. 

With a significance level of a = 0.05 (95% confidence level), use an ANOVA table to determine whether the multiple linear regression model determined in Example 3.28 is significant. 

Let y be a n X 1 response vector containing n measurements such that y = (yi, Y2,. .., Yn) - The p variables in the p x n matrix X will be referred to as predictors or independent variables, and the response vector y may be referred to as the dependent variable. We will assume that the predictor matrix and response vector have been appropriately centred prior to the regression analysis, thus ensuring the y-intercept term is zero. The general form of the multiple linear regression model is written as... 

The multiple linear regression model can also be described in terms of matrices as follows... 

Hence, R in the multiple linear regression model that predicts y from two independent predictor variables, Xi and X2, explains 75.3% or when adjusted, 73.9% of the variability in the model. The other 1 — 0.753 = 0.247 is xmexplained error. In addition, note that a fit of = 50% would infer that the prediction of y based on x and X2 is no better than y. 

PLSR is an extension of the multiple linear regression model. It is probably the least restrictive of the various multivariate extensions of the multiple linear regression model. This flexibility allows it to be used in situations where the use of traditional multivariate methods is severely limited, such as the case that when there are fewer observations than predictor variables. Furthermore, PLSR can be used as an exploratory analysis tool to select suitable predictor variables and to identify outliers before classical linear regression. Especially in chemometrics, PLSR has become a standard tool for modeling linear relationships between multivariate measurements. 

---