Non-linear PLS

The diffusion of correlation methods and related software packages, such as partial-least-squares regression (PLS), canonical correlation on principal components, target factor analysis and non-linear PLS, will open up new horizons to food research. 

Wang et al proposed a multivariate dominant factor based non-linearized PLS model for LIBS measurements. In constructing such a multivariate model, non-linear transformation of multi-characteristic line intensities according to the physical mechanisms of a laser-induced plasma spectrum were made, combined with a linear-correlation-based PLS method, to model the non-linear self-absorption and inter-element interference effects. Moreover, a secondary PLS was applied, utilizing information from the whole spectrum to correct the model results further. The proposed method showed a significant improvement when compared with a conventional PLS model. Even compared with the already improved baseline dominant-factor-based PLS model, the PLS model based on the multivariate dominant factor yielded the same calibration quality while decreasing the RMSEP. 

Z. Wang, J. Fenge, L. Li, W. Ni and Z. Li, A non-linearized PLS model based on multivariate dominant factor for laser-induced breakdown spectroscopy (LIBS) measurements, J. Anal. At. Spectrom., 2011, 26, 2175-2182. 

When any system or process is subjected to large changes, it appears nonlinear. In the present context, this means that the relation between X and Y becomes nonlinear. Also the relations between the X variables may become nonlinear, as may the relations between the Y variables. Even so, the X and Y matrices can always be approximated by the bilinear model (equations 6b and 3a). Hence, nonlinear situations can be modeled by non-linear PLS models, where the nonlinearities are expressed as nonlinear relations between the X scores and the Y scores Ua- These nonlinearities can be modeled as polynomial nonlinearities (quadratic, cubic, etc.), spline functions, or other nonlinear forms (e.g., bi-exponential). 

Additionally, Breiman et al. [23] developed a methodology known as classification and regression trees (CART), in which the data set is split repeatedly and a binary tree is grown. The way the tree is built, leads to the selection of boundaries parallel to certain variable axes. With highly correlated data, this is not necessarily the best solution and non-linear methods or methods based on latent variables have been proposed to perform the splitting. A combination between PLS (as a feature reduction method — see Sections 33.2.8 and 33.3) and CART was described by... 

In recent years there has been much activity to devise methods for multivariate calibration that take non-linearities into account. Artificial neural networks (Chapter 44) are well suited for modelling non-linear behaviour and they have been applied with success in the field of multivariate calibration [47,48]. A drawback of neural net models is that interpretation and visualization of the model is difficult. Several non-linear variants of PCR and PLS regression have been proposed. Conceptually, the simplest approach towards introducing non-linearity in the regression model is to augment the set of predictor variables (jt, X2, ) with their respective squared terms (xf,. ..) and, optionally, their possible cross-product... 

Now, what is interesting about this situation is that ordinary regression theory and the theory of PCA and PLS specify that the model generated must be linear in the coefficients. Nothing is specified about the nature of the data (except that it be noise-free, as our simulated data is) the data may be non-linear to any degree. Ordinarily this is not a problem because any data transform may be used to linearize the data, if that is desirable. 

PLS should have, in principle, rejected a portion of the non-linear variance resulting in a better, although not completely exact, fit to the data with just 1 factor. The PLS does tend to reject (exclude) those portions of the x-data which do not correlate linearly to the y-block. (Richard Kramer)... 

PLS should have, in principle, rejected a portion of the non-linear variance resulting in a better, although not completely exact, fit to the data with just 1 factor. 

Nonlinearity is a subject the specifics of which are not prolifically or extensively discussed as a specific topic in the multivariate calibration literature, to say the least. Textbooks routinely cover the issues of multiple linear regression and nonlinearity, but do not cover the issue with full-spectrum methods such as PCR and PLS. Some discussion does exist relative to multiple linear regression, for example in Chemometrics A Textbook by D.L. Massart et al. [6], see Section 2.1, Linear Regression (pp. 167-175) and Section 2.2, Non-linear Regression, (pp. 175-181). The authors state,... 

Our data indicate that the stars become fainter as metallicity increases, until a plateau or turnover point is reached at about solar metallicity. Our data are incompatible with both no dependence of th PL relation on iron abundance and with the linearly decreasing behaviour often found in the literature (e.g. [5], [8]). On the other hand, non-linear theoretical models of [2] provide a fairly good description of the data. For an in-depth discussion see [7]. 

M. Blanco, J. CoeUo, H. Iturriaga, S. Maspoch and J. Pages, NIR calibration in non-linear systems different PLS approaches and artificial neural networks, Chemom. Intell. Lab. Syst, 50, 75-82 (2000). 

This result could be an indicator of the improved ability of the ANN method to model non-linear relationships between the X-data and the 7-data. It could be the case that one of the four PLS latent variables is used primarily to account for such non-linearities, whereas the ANN method can more efficiently account for these non-linearities through the non-linear transfer function in its hidden layer. 

It could be the case that the specific method that was used to develop the model required assumptions about the calibration data that were highly inaccurate. An example of such a case would be that of the CLS method applied to a set of data where all of the chemical constituents in the samples were not known. Or, the MLR, PCR, or PLS methods were used on a set of data where strong non-linearities between the property of interest and many of the X-variables exist. In such cases, the modeling method itself is ill suited to provide an optimal model. 

It should be noted, however, that the factor-based methods (PCR and PLS) have the ability to model weak non-linearities rather well. Consequently, the model errors due to non-linearity might not be large enough to justify a considerable increase in model complexity through the use of non-linear modeling and subset modeling methods. Therefore, the decision to change the model structure is very application-dependent. 

Blanco, M., Coello, J., Iturriaga, H., Maspoch, S. and Pages, J., NIR Calibration in Non-linear Systems Different PLS Approaches and Artificial Neural Networks Chemometrics Intell. Lab. Syst. 2000, 50, 75-82. 

In Fig. 10, the transients exhibit quite different behavior from opal A to opal CT. In particular, a bi-exponential decay (Eq. 2) failed to reproduce the kinetics of opal CT. In this material, the emission is red-shifted towards 2.6 eV and the PL is strongly quenched at shorter time delays, with an unusual, non-linear kinetics in semi-log scale, indicating a complex decay channel either involving multi-exponential relaxation or exciton-exciton annihilations. Runge-Kutta integration of Eq. 5 seems to confirm the latter assumption with satisfactory reproduction of the observed decays. The lifetimes and annihilation rates are Tct = 9.3 ns, ta = 13.5 ns, 7ct o = 650 ps-1 and 7 0 = 241 ps-1, for opal CT and opal A, respectively. 

Non-linearities occur in some forms of spectroscopy, especially when the absorbance is high, and greater effort has been made to enhance the basic PLS method to include squared and other terms. However, the analytical chemist will probably prefer to improve the experimental method of acquiring data. Non-linear calibration is most valuable in other areas of chemistry, such as QSAR, where a purely additive linear model is not necessarily expected. 

Recently, PLS modelling involving non-linear inner relation has been described [76]. 

Predictive models are built with ANN s in much the same way as they are with MLR and PLS methods descriptors and experimental data are used to fit (or train in machine-learning nomenclature) the parameters of the functions until the performance error is minimized. Neural networks differ from the previous two methods in that (1) the sigmoidal shapes of the neurons output equations better allow them to model non-linear systems and (2) they are subsymbolic , which is to say that the information in the descriptors is effectively scrambled once the internal weights and thresholds of the neurons are trained, making it difficult to examine the final equations to interpret the influences of the descriptors on the property of interest. 

---