Eigenvalue decomposition

The power algorithm [21] is the simplest iterative method for the calculation of latent vectors and latent values from a square symmetric matrix. In contrast to NIPALS, which produces an orthogonal decomposition of a rectangular data table X, the power algorithm decomposes a square symmetric matrix of cross-products X which we denote by C. Note that Cp is called the column-variance-covariance matrix when the data in X are column-centered. 

In the power algorithm one first computes the matrix product of Cp with an initial vector of p random numbers v, yielding the vector w  

The result is then normalized, which produces an updated vector v  

The normalization step prevents the elements in v from becoming either too large or too small during the numerical computation. The two operations above define the cycle of the powering algorithm, which can be iterated until convergence of the elements in the vector v within a predefined tolerance. It can be easily shown that after n iterations the resulting vector w can be expressed as  

A key operation in the power algorithm is the calculation of the deflated cross-product matrix which is independent of the contribution by the first eigenvector. This is achieved by means of the instmction  

---

Thus far we have considered the eigenvalue decomposition of a symmetric matrix which is of full rank, i.e. which is positive definite. In the more general case of a symmetric positive semi-definite pxp matrix A we will obtain r positive eigenvalues where r general case we obtain a pxr matrix of eigenvectors V such that ... 

For the previously defined 2x2 matrix A we obtain the inverse from its eigenvalue decomposition, which has already been derived ... 

Singularity of the matrix A occurs when one or more of the eigenvalues are zero, such as occurs if linear dependences exist between the p rows or columns of A. From the geometrical interpretation it can be readily seen that the determinant of a singular matrix must be zero and that under this condition, the volume of the pattern P" has collapsed along one or more dimensions of SP. Applications of eigenvalue decomposition of dispersion matrices are discussed in more detail in Chapter 31 from the perspective of data analysis. 

From the 4x2 matrix X of our previous illustration we already derived V and from the eigenvalue decomposition of the 2x2 cross-product matrix X X ... 

In a similar way we can derive the eigenvalue decomposition of the corresponding 4x4 cross-product matrix XX ... 

Equation (31.3) defines the eigenvalue decomposition (EVD), also referred to as spectral decomposition, of a square symmetric matrix. The orthonormal matrices U and V are the same as those defined above with SVD, apart from the algebraic sign of the columns. As pointed out already in Section 17.6.1, the diagonal matrix can be derived from A simply by squaring the elements on the main diagonal of A. 

For the sake of completeness we mention here an alternative definition of eigenvalue decomposition in terms of a constrained maximization problem which can be solved by the method of Lagrange multipliers ... 

Fig. 31.13. Schematic example of three common algorithms for singular value and eigenvalue decomposition.

A comparison of the performance of the three algorithms for eigenvalue decomposition has been made on a PC (IBM AT) equipped with a mathematical coprocessor [38]. The results which are displayed in Fig. 31.14 show that the Householder-QR algorithm outperforms Jacobi s by a factor of about 4 and is superior to the power method by a factor of about 20. The time for diagonalization of a square symmetric value required by Householder-QR increases with the power 2.6 of the dimension of the matrix. 

Fig. 31.14. Performance of three computer algorithms for eigenvalue decomposition as a function of the dimension of the input matrix. The horizontal and vertical scales are scaled logarithmically. Execution time is proportional to a power 2.6 of the dimension.

Once the nxn variance-covariance matrix C has been derived one can apply eigenvalue decomposition (EVD) as explained in Section 31.4.2. In this case we obtain ... 

Correspondence factor analysis can be described in three steps. First, one applies a transformation to the data which involves one of the three types of closure that have been described in the previous section. This step also defines two vectors of weight coefficients, one for each of the two dual spaces. The second step comprises a generalization of the usual singular value decomposition (SVD) or eigenvalue decomposition (EVD) to the case of weighted metrics. In the third and last step, one constructs a biplot for the geometrical representation of the rows and columns in a low-dimensional space of latent vectors. 

In practice, the solution of Equation 3.16 for the estimation of the parameters is not done by computing the inverse of matrix A. Instead, any good linear equation solver should be employed. Our preference is to perform first an eigenvalue decomposition of the real symmetric matrix A which provides significant additional information about potential ill-conditioning of the parameter estimation problem (see Chapter 8). 

According to Scales (1985) the best way to solve Equation 5.12b is by performing a Cholesky factorization of the Hessian matrix. One may also perform a Gauss-Jordan elimination method (Press et al., 1992). An excellent user-oriented presentation of solution methods is provided by Lawson and Hanson (1974). We prefer to perform an eigenvalue decomposition as discussed in Chapter 8. 

The eigenvalue decomposition of the positive definite symmetric matrix A... 

A more interesting interpretation of Levenberg-Marquardt s modification can be obtained by examining the eigenvalues of the modified matrix (A+y2 ). If we consider the eigenvalue decomposition of A, V1 AV we have,... 

Step 4. Perform an eigenvalue decomposition of A=VTAV and compute A -VtA V. 

Again, we can determine the condition number and X,nin of matrix Anew using any eigenvalue decomposition routine that computes the eigenvalues of a real symmetric matrix and use the conditions (xN+0 that correspond to a maximum of... 

Correlation between three or more parameters is very difficult to detect unless an eigenvalue decomposition of matrix A is performed. As already discussed in Chapter 8, matrix A is symmetric and hence an eigenvalue decomposition is also an orthogonal decomposition... 

After performing the eigenvalue decomposition of matrix the vector p of principal components of is calculated through Eq. (11.37). The elements of p are represented in Fig. 17. 

The quadratic model (Eq.3.3) allowed the generation of the 3-D response surface image (Fig. 3.5) for the main interaction between injection time and voltage. The quadratic terms in this equation models the curvature in the true response function. The shape and orientation of the curvature results from the eigenvalue decomposition of the matrix of second-order parameter estimates. After the parameters are estimated, critical values for the factors in the estimated surface can be found. For this study, a post hoc review of our model... 

---