Linear Spectral Dimensionality Reduction

Linear approaches to spectral dimensionality reduction make the assumption that the data lies on or near a low-dimensional subspace. In such cases, linear spectral dimensionality reduction methods seek to learn the basis vectors of this low-dimensional subspace so that the input data can be projected onto the linear subspace. The two main methods for linear spectral dimensionality reduction. Principal Components Analysis and Multidimensional Scaling, are both described in this section. Although more powerful nonlinear approaches have been presented in recent years, these linear techniques are still widely used and are worthy of attention since they provide the basis for some of the subsequent nonlinear spectral dimensionality reduction algorithms. 

---

This section begins by providing a general mathematical setting within which both spectral dimensionality reduction, and the associated open problems, can be described. Then key algorithms, both linear and nonlinear, are briefly described so as to provide an important point of reference and discussion for the later discussion of open problems. 

Spectral dimensionality reduction seeks to obtain a low-dimensional embedding of a high-dimensional dataset through the eigendecomposition of a specially constructed feature matrix. This feature matrix will capture certain properties of the data such as inter-point covariance or local linear reconstruction weights. The different methods of formulating this feature matrix will have different implications for various open problems, as will be seen in later chapters. 

Spectroscopic methods are increasingly employed for quantitative applications in many different fields, including chemistry [1]. The dimensionality of spectral data sets is basically limited by the number of the objects studied, whereas the number of variables can easily reach a few thousands. Highdimensional spectral data are very correlated and usually somewhat noisy, so that, the conventional multiple linear regression (MLR) cannot be applied to this type of data directly the feature selection or reduction procedures are needed [2],... 

---