NIPALS algorithm

In this way, given U one can compute V, and vice versa. These transition formulae are the basis for the calculation of U and V by the so-called NIPALS algorithm which will be explained in Section 31.4. 

Fig. 31.12. Dance-step diagram, illustrating a cycle of the iterative NIPALS algorithm. Step 1 multiplies the score vector t with the data table X, which produces the weight veetor w. Step 2 normalizes w to unit sum of squares. In step 3, X is multiplied by w, yielding an updated t.

A crucial operation in the NIPALS algorithm is the calculation of the residual data matrix which is independent of the contributions by the first singular vector. This can be produced by the instruction ... 

The NIPALS algorithm is easy to program, particularly with a matrix-oriented computer notation, and is highly efficient when only a few latent vectors are required, such as for the construction of a two-dimensional biplot. It is also suitable for implementation in personal or portable computers with limited hardware resources. 

H. Wold, Soft modelling by latent variables the non-linear iterative partial least squares (NIPALS) algorithm. In Perspectives in Probability and Statistics, J. Gani (Ed.). Academic Press, London, 1975, pp. 117-142. 

PLS has been introduced in the chemometrics literature as an algorithm with the claim that it finds simultaneously important and related components of X and of Y. Hence the alternative explanation of the acronym PLS Projection to Latent Structure. The PLS factors can loosely be seen as modified principal components. The deviation from the PCA factors is needed to improve the correlation at the cost of some decrease in the variance of the factors. The PLS algorithm effectively mixes two PCA computations, one for X and one for Y, using the NIPALS algorithm. It is assumed that X and Y have been column-centred as usual. The basic NIPALS algorithm can best be demonstrated as an easy way to calculate the singular vectors of a matrix, viz. via the simple iterative sequence (see Section 31.4.1) ... 

Instead of separately calculating the principal components for each data set, the two iterative sequences are interspersed in the PLS-NIPALS algorithm (see Fig. 

Slightly different implementations of the above PLS-NIPALS algorithm exist. They mostly differ in the chosen normalization of w, t or p (here Iwl = 1). This is not an important issue, but it may be a cause of confusion when comparing results from different (software) implementations. That the normalization is of no real importance can be seen as follows. Let us say we choose to multiply the weight... 

A slight alternative to the above NIPALS algorithm is replacing the iteration loop (line 8-14) by ... 

The nonlinear iterative partial least-squares (NIPALS) algorithm, also called power method, has been popular especially in the early time of PCA applications in chemistry an extended version is used in PLS regression. The algorithm is efficient if only a few PCA components are required because the components are calculated step-by-step. 

FIGURE 3.12 NIPALS algorithm for PCA. Left scheme (a) shows the iterative procedure for calculation of a PC. The right scheme (b) describes the peeling process (deflation) for elimination of the information of a PC. 

In 1 a function has been provided for an easy application of the NIPALS algorithm for the calculation of, for instance, two PCs of a mean-centered matrix X, the R-code is as follows ... 

The NIPALS algorithm is efficient if only a few PCA components are required. Because the deflation procedure increases the uncertainty of following components, the algorithm is not recommended for the computation of many components (Seasholtz et al. 1990). The algorithm fails if convergence is reached already after one cycle in this case another initial value of the score vector has to be tried (Miyashita et al. 1990). 

We describe the most used version with the notation used in the previous section. The main steps of the NIPALS algorithm are as follows. Suppose we want to find the first PLS component, then the pseudocode is... 

For subsequent PLS components, the NIPALS algorithm works differently than the kernel method however, the results are identical. NIPALS requires a deflation of X and of Y and the above pseudocode is continued by... 

For PLS1 regression, the NIPALS algorithm simplifies. It is no longer necessary to use iterations for deriving one PLS component. Thus the complete pseudocode for extracting a components is as follows ... 

Calculation of eigenvectors requires an iterative procedure. The traditional method for the calculation of eigenvectors is Jacobi rotation (Section 3.6.2). Another method—easy to program—is the NIPALS algorithm (Section 3.6.4). In most software products, singular value decomposition (SVD), see Sections A.2.7 and 3.6.3, is applied. The example in Figure A.2.7 can be performed in R as follows ... 

There are several different PLS algorithms, the most common of which are the NIPALS algorithm [1,64], the SIMPLS algorithm [65], and the Bidiagonalization algorithm [66]. These algorithms are somewhat more... 

Like PCR, a PLS model can be condensed into a set of regression coefficients (bpLs)- For the NIPALS algorithm discnssed above, the regression coefficients can be calculated by the following equation ... 

Figure 3. NIPALS algorithm for extraction of principal conponents from a data matrix.

The decomposition in eqn (3.30) is general for PCR, PLS and other regression methods. These methods differ in the criterion (and the algorithm) used for calculating P and, hence, they characterise the samples by different scores T. In PCR, T and P are found from the PCA of the data matrix R. Both the NIPALS algorithm [3] and the singular-value decomposition (SVD) (much used, see Appendix) of R can be used to obtain the T and P used in PCA/PCR. In PLS, other algorithms are used to obtain T and P (see Chapter 4). 

Figure 4.3 Step 1 of the NIPALS algorithm mean centring, starting point and first calculations.

Figure 4.4 Step 2 of the NIPALS algorithm scores for the X-block, taking into account the information in the concentrations (Y-block).

---