Nonlinear projection methods

Alternate projection methods aim to reduce the data dimensionality by optimizing the representation in the lower-dimension space so that the distances between points in the projected space are as similar as possible to the distances between the corresponding points in the original space. We will describe here a class of methods known as multidimensional scaling (MDS). The aim of these methods is to project data from a pseudo-metric space (i.e., one characterized by a dissimilarity measure) onto a metric space. Such methods are especially useful for preprocessing non-metric data in order to use algorithms valid only for metric input. 

The first MDS method is the metric MDS, characterized by minimizing the squared error cost function  

Another nonlinear mapping method, the Sammon s mapping, is closely related to the metric MDS. The only difference is that the errors in distance preservation are normalized with the distance in the original space. Thus, preservation of small distances is emphasized. The error function is defined as 

---

While PCA is a linear projection method, there also exist nonlinear projection methods, e.g. multidimensional scaling [Mardia et al. 1979] and nonlinear PCA [Dong McAvoy 1996], A good overview of nonlinear multivariate analysis tools is given by [Gift 1990],... 

Methods based on nonlinear projection exploit the nonlinear relationship between the inputs by projecting them on a nonlinear hypersurface resulting in latent variables that are nonlinear functions of the inputs, as shown in Figs. 6b and 6c. If the inputs are projected on a localized hypersurface such as a hypersphere or hyperellipse, then the basis functions are local, depicted in Fig. 6c. Otherwise, the basis functions are nonlocal, as shown in Fig. 6b. 

Fig. 6. Input transformation in (a) methods based on linear projection, (b) methods based on nonlinear projection, nonlocal transformation, (c) methods based on nonlinear projection, local transformation, and (d) partition-based methods. (From Bakshi and Utojo, 1998.)...

Techniques for multivariate input analysis reduce the data dimensionality by projecting the variables on a linear or nonlinear hypersurface and then describe the input data with a smaller number of attributes of the hypersurface. Among the most popular methods based on linear projection is principal component analysis (PCA). Those based on nonlinear projection are nonlinear PCA (NLPCA) and clustering methods. 

Methods based on nonlinear projection are distinguished from the linear projection methods that they transform input data by projection on a nonlin-... 

Data interpretation methods can be categorized in terms of whether the input space is separated into different classes by local or nonlocal boundaries. Nonlocal methods include those based on linear and nonlinear projection, such as PLS and BPN. The class boundary determined by these methods is unbounded in at least one direction. Local methods include probabilistic methods based on the probability distribution of the data and various clustering methods when the distribution is not known a priori. 

The constrained minimization problem stated above may be transformed into a form well-suited to gradient projection methods of nonlinear programming by making the following substitution ... 

Mulligan, A. E., and Ahlfeld, D. P. (1999b). An interior point boundary projection method for nonlinear groundwater optimization with zero-derivative constraints. RCGRD Publication 98-2, University ofVermont, Burlington, VT. 

---