Biased regression

Mallows Cp. Model selection criterion used to compare biased regression models with the full least squares regression model ... 

Partial least squares (PLS) regression, develops a biased regression model between X and Y. In the context of chemical process operations, usually X denotes the process variables and Y the quality variables. PLS selects latent variables so that variation in X which is most predictive of the product quality data Y is extracted. PLS works on the sample covariance matrix (X Y)(Y X) [86, 87, 111, 172, 188, 334, 338[. Measurements of m process variables taken at n different times are arranged into a (n x m) process data matrix X. The q quality variables are given by the corresponding... 

Principal components regression is another biased regression technique but when done successfully is superior to OLS in terms of prediction and estimation. Principal components (PC) are linear transformations of the original variables such that each PC is orthogonal or uncorrelated to the others (Jackson, 1991). There will be k principal components if there are k variables. Of these k principal components, j (j < k) components may contain most of the information contained in k. Thus, regression of the j principal components, instead of the original k variables, may be used for regression. The... 

Iwatsubo et al., 1996 Iwatsubo et al., 1997), or the estimation of drug clearance based on creatinine clearance (Bazunga et al., 1998 Lefevre et al., 1997). In these three examples, log P, in vitro clearance, and creatinine clearance, all have some measurement error associated with them that may be large enough to produce significantly biased regression parameter estimates. 

PCR and PLS are examples of biased regression methods where the expected estimated value might differ from the true value of the parameter. Another powerful biased regression method is ridge regression. 

In which situations are biased regression methods particularly useful compared to ordinary linear regression analysis ... 

Although Eq. [29] has the appearance of a multiple regression, remember that the parameter estimates were not calculated by OLS. Instead they were found by a biased regression method. Consequently, these parameters, which are referred to as pseudo-p s, will not in general equal the OLS values because they have been shrunken, (some more than others). However, as more components are added into the latent variable model (Eq. [27]), i.e., as p increases toward k, these pseudo-p s approach the values obtained by OLS. In the limit p = k, Eq. [29] will be identical to the OLS model, a result that will be illustrated later when we apply the methods to a real dataset. 

Locally biased regression [13] identifies samples similar to the unknown using distance in a two-dimensional space. One axis of this space is a prediction of the analyte using aglobal PLS calibration. 

T. Fearn and A.M.C. Davies, Locally-biased regression. J. Near Infrared Spectrosc. 11, 467 (2003). 

To overcome these problems, MacGregor et al. (1991) have looked at biased regression techniques (e.g. ridge regression (RR)) and the projection to latent structures (PLS) method as alternatives to least squares. Ricker (1988) studied the use of PLS and a method based on the singular value decomposition (SVD). All of these approaches attempt to reduce the parameter variances and improve the numerical stability of the solution with the tradeoff being biased models. 

---