Reduced rank regression

Reduced rank regression (RRR), also known as redundancy analysis (or PCA on Instrumental Variables), is the combination of multivariate least squares regression and dimension reduction [7]. The idea is that more often than not the dependent K-variables will be correlated. A principal component analysis of Y might indicate that A (A m) PCs may explain Y adequately. Thus, a full set of m 

The question of how many components to include in the final model forms a rather general problem that also occurs with the other techniques discussed in this chapter. We will discuss this important issue in the chapter on multivariate calibration. 

An alternative and illuminating explanation of reduced rank regression is through a principal component analysis of Y, the set of fitted F-variables resulting from an unrestricted multivariate multiple regression. This interpretation reveals the two least-squares approximations involved projection (regression) of Y onto X, followed by a further projection (PCA) onto a lower dimensional subspace. 

The interpretation also suggests the following simple computational implementation of reduced rank regression. 

Equation (35.15) represents the projection of each K-variable onto the space spanned by the X-variables, i.e. each 7-variable is replaced by its fit from multiple regression on X. 

---

H. Schmidli. Reduced Rank Regression. Physica-Verlag, 1995. 

Section 35.4), reduced rank regression (Section 35.5), principal components regression (Section 35.6), partial least squares regression (Section 35.7) and continuum regression methods (Section 35.8). 

Fig. 35.5. Biplot of reduced rank regression model showing objects, predictors and responses.

P.T. Davies and M.K.S. Tso, Procedures for reduced-rank regression. Appl. Stat., 31 (1982)... 

Subspace modeling can be cast as a reduced rank regression (RRR) of collections of future outputs on past inputs and outputs after removing the effects of future inputs. CVA performs this regression. In the case of a linear system, an approximate Kalman filter sequence is recovered from this regression. The state-space coefficient matrices are recovered from the state sequence. The nonlinear approach extends this regression to allow for possible nonlinear transformations of the past inputs and outputs, and future inputs and outputs before RRR is performed. The model structure consists of two sub models. The first model is a multivariable dynamic model for a set of latent variables, the second relates these latent variables to outputs. The latent variables are linear combinations of nonlinear transformations of past inputs and outputs. These nonlinear transformations or functions are... 

---