Nonlinear PLS

S. Wold, N. Kettaneh-Wold and B.Skagerberg, Nonlinear PLS modeling. Chemom. Intell. Lab. Syst., 7 (1989) 53-65. 

Similarly, extending linear PLS to nonlinear PLS involves using nonlinear basis functions. A variety of nonlinear basis functions have been used to model the inner relationship indicated in Eq. (22), including quadratic... 

Wold, S., Nonlinear PLS modeling II spline inner relation, Chemom. Intell. Lab. Sys., 14, 71-84 (1992). 

All algorithms described so far solve the linear PLS regression models (Equations 4.61 through 4.65). For Nonlinear PLS we assume nonlinear relations, and two major approaches are mentioned here (Rosipal and Kramer 2006). 

Among the nonlinear methods, there are, besides nonlinear least squares regression, i.e. polynomial regression, the nonlinear PLS method. Alternating Conditional Expectations ACE), SMART, and MARS. Moreover, some Artificial Neural Networks techniques have also to be considered among nonlinear regression methods, such as the back-propagation method. 

Other model alternatives are higher order polynomials, rational functions of several variables, nonlinear PLS, neural networks, nonlinear SVM etc. With higher order polynomials, or with linearized rational functions, it advisable to use ridge regression, PLS, or some other constrained regression technique, see e.g. (Taavitsainen, 2010). These alternatives are useful typically in cases where the response is bounded in the experimental region see e.g. (Taavitsainen et. al., 2010). 

The variables (wavelengths) associated with the IR emission spectra were highly correlated. Principal components analysis (PCA), linear and nonlinear PLS showed that at least 86% of the total variance could be explained by the two primary latent dimensions. The forward and reverse modelling results showed that dimensional reduction with a linear model (PLS) produced better models than a nonlinear model (multilayer perceptron neural network trained with the back propagation algorithm) without dimensional reduction. 

Wold, S., Kettaneh-Wold, N. Skagerberg, B. (1998). Nonlinear PLS modeling, Chemometrics and Intelligent Laboratory Systems, Vol. 7, pp. 53-65. 

No quantitative applications of neural networks to quantitative analysis of linear systems have been reported where the results have been significantly better than those obtained by PLS, as would be expected since PLS (and indeed, the other multivariate methods described in this chapter) have been designed explicitly to handle linear systems. Analogous techniques, such as polynomial PLS or spline PLS, have been designed for nonlinear systems. It is interesting that with nonlinear data, neural networks have been shown to outperform any of the linear or nonlinear PLS techniques [27]. 

---