Feature space reduction methods

Inappropriate choice of feature space reduction methods... 

A. Nikulin, B. Dolenko, T. Bezebah, R.L. Somorjai, Near-optimal Region Selection for Feature Space Reduction Novel Preprocessing Methods for Classifying MR Spectra , NMR Biomed., 11, 209-216 (1998). 

Similar to PCA, LDA is a feature reduction method. For this purpose, a 1-dimensional space, that is, a line, on which the objects will be projected from... 

PCA is a method based on the Karhunen-Loeve transformation (KL transformation) of the data points in the feature space. In KL transformation, the data points in the feature space are rotated such that the new coordinates of the sample points become the linear combination of the original coordinates. And the first principal component is chosen to be the direction with largest variation of the distribution of sample points. After the KL transformation and the neglect of the components with minor variation of coordinates of sample points, we can make dimension reduction without significant loss of the information about the distribution of sample points in the feature space. Up to now PCA is probably the most widespread multivariate statistical technique used in chemometrics. Within the chemical community the first major application of PCA was reported in 1970s, and form the foundation of many modem chemometric methods. Conventional approaches are univariate in which only one independent variable is used per sample, but this misses much information for the multivariate problem of SAR, in which many descriptors are available on a number of candidate compounds. PCA is one of several multivariate methods that allow us to explore patterns in multivariate data, answering questions about similarity and classification of samples on the basis of projection based on principal components. 

Many methods have been developed to tackle the issue of high dimensionality of hyperspectral data (Serpico and Bruzzone 1994). In summary, we may say that feature-reduction methods can be divided into two classes feature-selection algorithms (which suitably select a suboptimal subset of the original set of features while discarding the remaining ones) and feature extraction by data transformation which projects the original data space onto a lower-dimensional feature subspace that preserves most of the information, such as nonlinear principal component analysis (NLPCA Licciardi and Del Prate 2011). 

Isomap [2], one of the first true nonlinear spectral dimensionality reduction methods, extends metric MDS to handle nonlinear manifolds. Whereas metric MDS measures inter-point EucUdean distances to obtain a feature matrix. Isomap measures the interpoint manifold distances by approximating geodesics. The use of manifold distances can often lead to a more accurate and robust measure of distances between points so that points that are far away according to manifold distances, as measured in the high-dimensional space, are mapped as far away in the low-dimensional space (Fig. 2.3). An example low-dimensional embedding of the S-Curve dataset (Fig. 2.1) found using Isomap is given in Fig. 2.4. 

A more sophisticated form of reduction is obtained when the so-called center manifold theorem is invoked. This says essentially that a subspace of lower dimension than the whole state-space gives a true representation of the essential features of the system and one that can be built on to give a yet more accurate picture. We shall not attempt to go into this here to see the method in action, the reader cannot do better than to read C. Chang and... 

MEISs and macroscopic kinetics. Formalization of constraints on chemical kinetics and transfer processes. Reduction of initial equations determining the limiting rates of processes. Development of the formalization methods of kinetic constraints direct application of kinetics equations, transition from the kinetic to the thermodynamic space, and direct setting of thermodynamic constraints on individual stages of the studied process. Specific features of description of constraints on motion of the ideal and nonideal fluids, heat and mass exchange, transfer of electric charges, radiation, and cross effects. Physicochemical and computational analysis of MEISs with kinetic constraints and the spheres of their effective application. 

The number of features combined in a vector-type representation is indicative of the dimensionality of the problem space. Low-dimensional representations, on the one hand, allow easy visualization but are most often not very discriminative. Highdimensional representations, on the other hand, such as those encoded in Daylight fingerprints [23], MACCS keys [24], or UNITY fingerprints [25], provide more detailed accounts on structural or chemical variations. However, this is achieved at the cost of visualization. Part of these high-dimensional representations describe specific local features of molecules, and because not all molecules in the data contain these features, gaps or zeros are introduced in the data representation. For certain data mining methods, this could be problematic. In many cases, dimensionality reduction procedures are applied to reduce the complexity of the representation. The reduction of the dimensionality is accomplished by means of 1) variable selection procedures, 2)... 

---