PLS components

The most predictive PLS regression model for these data makes use of two PLS-components ... 

The selection of the number of PLS-components to be included in the model was done according to the PRESS criterion (Section 36.3). Note that the result is comparable to the one which we obtained earlier by means of the simple Hansch analysis (Section 37.1.1). Hence, in this case, there is no obvious benefit to include a quadratic term of log P in the model. 

The number of latent variables (PLS components) must be determined by some sort of validation technique, e.g., cross-validation [42], The PLS solution will coincide with the corresponding MLR solution when the number of latent variables becomes equal to the number of descriptors used in the analysis. The validation technique, at the same time, also serves the purpose to avoid overfitting of the model. 

In double CV, the CV strategy is applied in an outer loop (outer CV) to split all data into test sets and calibration sets, and in an inner loop (inner CV) to split the calibration set into training sets and validation sets (Figure 4.6). The inner loop is used to optimize the complexity of the model (for instance, the optimum number of PLS components as shown in Figure 4.5) the outer loop gives predicted values yjEST for all n objects, and from these data a reasonable estimation of the prediction performance for new cases can be derived (for instance, the SEPtest). It is important... 

The number of segments in the outer and inner loop (. 0ut and sin, respectively) may be different. Each loop of the outer CV results in an optimum complexity (for instance, optimum number of PLS components, Aopt)- In general, these Sout values are different for a final model the median of these values or the most frequent value can be chosen (a smaller complexity would avoid overfitting, a larger complexity would result in a more detailed model but with the risk of overfitting). A final model can be created from all n objects applying the final optimum complexity the prediction performance of this model has been estimated already by double CV. This strategy is especially useful for PLS and PCR. 

During model development, a relatively small number of PLS components (intermediate linear latent variables) are calculated which are internally used for regression. 

The number of PLS components determines the complexity of the model and can be optimized for high prediction performance. 

First PLS-component is calculated as the latent variable which has maximum covariance between the scores and modeled property y. Note that the criterion covariance is a compromise between maximum correlation coefficient (OLS) and maximum variance (PCA). 

From the residual matrix, the next PLS component is derived—again with maximum covariance between the scores and y. 

A complicating aspect of most PLS algorithms is the stepwise calculation of the components. After a component is computed, the residual matrices for X (and eventually Y) are determined. The next PLS component is calculated from the residual matrices and therefore its parameters (scores, loadings, weights) do not relate to X but to the residual matrices. However, equations exist, that relate the PLS -x-loadings and PLS-x-scores to the original x-data, and that also provide... 

The first PLS component is found as follows Since we deal with the sample covariance, the maximization problem (Equation 4.67) can be written as maximization of... 

Further PLS components (t2, Pi. and so on) are obtained by the same algorithm as the first components using the deflated X matrix obtained after calculation of the previous component. The procedure is continued until a components have been extracted. 

We describe the most used version with the notation used in the previous section. The main steps of the NIPALS algorithm are as follows. Suppose we want to find the first PLS component, then the pseudocode is... 

For subsequent PLS components, the NIPALS algorithm works differently than the kernel method however, the results are identical. NIPALS requires a deflation of X and of Y and the above pseudocode is continued by... 

Steps 11-13 are the OLS estimates using the regression models (Equations 4.62 through 4.64). Step 14 performs a deflation of the X and of the Y matrix. The residual matrices Xt and Yi are then used to derive the next PLS components, following the scheme of steps 1-10. Finally, the regression coefficients B from Equation 4.61 linking the y-data with the x-data are obtained by B = Y(P Y) C ... 

For PLS1 regression, the NIPALS algorithm simplifies. It is no longer necessary to use iterations for deriving one PLS component. Thus the complete pseudocode for extracting a components is as follows ... 

FIGURE 4.36 Robust PLS for the PAC data set using 21 PLS components. The plots show measured versus predicted values (left) and predicted values versus residuals (right) using a single 10-fold CV. 

FIGURE 4.41 Stepwise regression for the PAC data set. The BIC measure is reduced within each step of the procedure, resulting in models with a certain number of variables (left). The evaluation of the final model is based on PLS where the number of PLS components is determined by repeated double CV (right). 

For model validation we use leave-one-out CV. Other validation schemes could be used, but in this example we have a severe limitation due to the low number of objects. A plot of the prediction errors from CV versus number of PLS components is shown in Figure 4.43. The dashed lines correspond to the MSE values for the... 

FIGURE 4.43 Prediction errors for the training data (dashed lines) and for the test data (solid lines, leave-one-out CV) of the cereal data set in relation to the number of PLS components. 

A decision on the number of PLS components can be made by plotting the averages of the MSEP (and MSE) values over all y-variables. This plot in Figure 4.44... 

FIGURE 5.29 Mean squared errors for different numbers of PLS components used for D-PLS. 

---