Full spectrum methods

The main problem was to define the conditions where wavelength selection was superior we could never quite put our finger on what characteristics of spectra would allow the wavelength-based techniques to perform better than full-spectrum methods. 

So we seem to have identified a key characteristic of chemometric modeling that influences the capabilities of the models that can be achieved not nonlinearity per se, because simple nonlinearity could be accommodated by a suitable transformation of the data, but differential nonlinearity, which cannot be fixed that way. In those cases where this type of differential, or non-uniform, nonlinearity is an important characteristic of the data, then selecting those wavelengths and only those wavelengths where the data are most nearly linear will provide better models than the full-spectrum methods, which are forced to include the non-linear regions as well, are capable of. 

While I am no longer working in this field, and cannot easily do simulations, I think that a 2 factor PCR or PLS model would fully model the simulated spectra. At any wavelength in your simulation, a second degree power series applies, which is linear in coefficients, and the coefficients of a 2 factor PCR or PLS model will be a linear function of the coefficients of the power series. (This assumes an adequate number of calibration spectra, that is, at least as many spectra as factors and a sufficient number of wavelength, which the full spectrum method assures.) The PCR or PLS regression should find the linear combination of these PCR/PLS coefficients that is linear in concentration. 

Nonlinearity is a subject the specifics of which are not prolifically or extensively discussed as a specific topic in the multivariate calibration literature, to say the least. Textbooks routinely cover the issues of multiple linear regression and nonlinearity, but do not cover the issue with full-spectrum methods such as PCR and PLS. Some discussion does exist relative to multiple linear regression, for example in Chemometrics A Textbook by D.L. Massart et al. [6], see Section 2.1, Linear Regression (pp. 167-175) and Section 2.2, Non-linear Regression, (pp. 175-181). The authors state,... 

FIGURE 5.SS. Demonstration of the error detection capabilities of the full-spectrum methods. The dashed line represents a spectrum that has a similar shape to the calibration data. The solid-line spectrum has a quite different shape than the calibration spectra. The vertical lines are two variables that have been selected for MLR. 

In the arsenal of calibration methods there are methods more suited for modelling any number of correlated variables. The most popular among them are Principal Component Regression (PCR) and Partial Least Squares (PLS) [3], Their models are based on a few orthogonal latent variables, each of them being a linear combination of all original variables. As all the information contained in the spectra can be used for the modelling, these methods are often called the full-spectrum methods. ... 

There are situations, when feature selection coupled with MLR can offer some advantages, compared to the full spectrum methods. This can happen, for instance, if there are many redundant X-variables with very different curvature in their relationship to Y. In such case, the feature selection procedure allows to eliminate those X-variables, which are most non-linear in their response, while their non-linear curvatures may contaminate the full-spectrum calibration models. 

Results of PLS, i.e. of the full spectrum method, are presented in Table 3. The RMSCV and RMSEP values are much higher than the analogous values observed for the SMLR or GA-MLR models, but one can hope that the PLS models are more stable e.g. when instrumental problems occur. Still, one can try to lower model complexity by extracting relevant information from the original spectra. This can be done, for instance, by using the UVE-PLS or... 

The PLS-2 technique is a typical full spectrum method where the data are fitted to many data points, thereby improving the precision and requires a carefully experimental design of the Standard composition of the calibration set order the provide good predictions. In this study training set of 27 representative ternary mixtures was constructed and the absorption spectra were recorded. In Table 33.1, the compositions of the ternary mixtures employed are summarized. 

Classical least-squares is a full spectrum method because all the digitized absorbances of the measured spectra are taken into account. A disadvantage of the CLS method is that all the interfering chemical components in the region of the spectrum being observed must be known and included in the analysis. Thus, spectral overlaps cause severe problems in this type of analysis unless explicitly taken into account. 

If there is particularly grave nonlinearity in the X-y relationships, and a surplus of X variables is available, then SMLR may sometimes give slightly better predictive ability than PLSR and other full-spectrum methods, since the SMLR can simply skip the X variables that display the gravest nonlinearity for y, while the full-spectrum methods may have to model them. 

On the other hand, if there is appreciable random noise in the NIR data (X), then full-spectrum methods must be expected to give better predictive precision than SMLR due to the full-spectrum smoothing effect. 

This full-spectrum method improves the precision over those that use only a few wave numbers. Corrections can be added for Beer s law deviations or fitting spectral baselines. However, all components present must be included in the calibration mixtures. 

In a variation of this type of analysis, the concentrations are expressed as functions of the various absorbances rather that vice-versa as before. This is called the inverse least squares (ILS) (or the Pmatrix method). An advantage of this method is that a quantitative analysis can be performed on some components using calibrated standards, even if some other components with unknown concentrations are present in the standards in amounts bracketing those in the samples. A disadvantage is that it is not a full-spectrum method. In the analysis, there must be at least as many standards for calibration as there are analytical wave numbers used. 

Two factor analysis methods that are used are the principal components regression (PCR) and the partial least squares (PLS). In the PRC method, the concentrations are expressed as functions of the principal components (PC) instead of absorbances as in ILS. The PC are orthogonal vectors that are linear combinations of the original spectral data of the standards. Here, PCI accounts for the maximum variability in the data, and PC2 accounts for the maximum variability not accounted for by PCI, etc. The other method PLS, is similar to the PCR method except that the PCs are weighted. The weighting is based on the correlation of the PCs with concentration. These are full-spectrum methods like CLS, but like ILS, one can analyze one component at a time. These methods are most often used for quantitative analysis in the near infrared region because of the broadness and overlapping nature of the bands here. 

---