Singular value diagonalization

Let u be a vector valued stochastic variable with dimension D x 1 and with covariance matrix Ru of size D x D. The key idea is to linearly transform all observation vectors, u , to new variables, z = W Uy, and then solve the optimization problem (1) where we replace u, by z . We choose the transformation so that the covariance matrix of z is diagonal and (more importantly) none if its eigenvalues are too close to zero. (Loosely speaking, the eigenvalues close to zero are those that are responsible for the large variance of the OLS-solution). In order to liiid the desired transformation, a singular value decomposition of /f is performed yielding... 

Here W is diagonal matrix of singular values, is the transpose of the second re-sultant matrix, being actually the same as the loading matrix in PCA, and X is the matrix, which is applied for further modeling. 

Singular value decomposition (SVD) of a rectangular matrix X is a method which yields at the same time a diagonal matrix of singular values A and the two matrices of singular vectors U and V such that ... 

In the previous section we have developed principal components analysis (PCA) from the fundamental theorem of singular value decomposition (SVD). In particular we have shown by means of eq. (31.1) how an nxp rectangular data matrix X can be decomposed into an nxr orthonormal matrix of row-latent vectors U, a pxr orthonormal matrix of column-latent vectors V and an rxr diagonal matrix of latent values A. Now we focus on the geometrical interpretation of this algebraic decomposition. 

S = 29.5803 0 0 0 1.9907 0 0 0 0.2038 Display the S matrix or the singular values matrix. This diagonal matrix contains the variance described by each principal component. Note the squares of the singular values are termed the eigenvalues. 

The important special properties of the three product matrices U, S and V are the following S is a diagonal matrix, containing the so-called singular values in descending order. Note that the singular values of real matrices are always positive and real. U and V are orthonormal matrices, which means they are comprised of orthonormal vectors. In matrix notation ... 

S diagonal matrix of significant singular values (nexne)... 

For mean-centered X the matrix To has size nxm and contains the PCA scores normalized to a length of 1. S is a diagonal matrix of size mxm containing the so-called singular values in its diagonal which are equal to the standard deviations of the scores. PT is the transposed PCA loading matrix with size mxm. The PCA scores, T. as defined above are calculated by... 

The use of singular value decomposition (SVD), introduced into chemical engineering by Moore and Downs Proc. JACC, paper WP-7C, 1981) can give some guidance in the question of what variables to control. They used SVD to select the best tray temperatures. SVD involves expressing the matrix of plant transfer function steady state gains as the product of three matrices a V matrix, a diagonal Z matrix, and a matrix. 

It is somewhat similar to canonical transformation. But it is different in that the diagonal 2 matrix contains as its diagonal elements, not the eigenvalues of the Kj, matrix, but its singular values. 

In Equation 4.13 we seek the k columns of U that are the column-mode eigenvectors of A. These k columns are the columns with the k largest diagonal elements of S, which are the square root of the eigenvalues of Z = ATA. The k rows of VT are the row-mode eigenvectors of A. The following equations describe the relationship between the singular-value decomposition model and the principal component model. 

Figure E-1 Singular value decomposition of a rectangular matrix A, using a column-orthogonal matrix U and a square diagonal matrix Q.

---