Column-singular vector

In Table 32.7 we observe a contrast (in the sense of difference) along the first row-singular vector u, between Clonazepam (0.750) and Lorazepam (-0.619). Similarly we observe a contrast along the first column-singular vector v, between epilepsy (0.762) and anxiety (-0.644). If we combine these two observations then we find that the first singular vector (expressed by both u, and v,) is dominated by the positive correspondence between Clonazepam and epilepsy and between Lorazepam and anxiety. Equivalently, the observations lead to a negative correspondence between Clonazepam and anxiety, and between Lorazepam and epilepsy. In a similar way we can interpret the second singular vector (expressed by both U2 and V2) in terms of positive correspondences between Triazolam and sleep and between Diazepam and anxiety. 

Note that the algebraic signs of the columns in U and V are arbitrary as they have been computed independently. In the above illustration, we have chosen the signs such as to be in agreement with the theoretical result. This problem does not occur in practical situations, when appropriate algorithms are used for singular vector decomposition. 

The eigenvectors extracted from the cross-product matrices or the singular vectors derived from the data matrix play an important role in multivariate data analysis. They account for a maximum of the variance in the data and they can be likened to the principal axes (of inertia) through the patterns of points that represent the rows and columns of the data matrix [10]. These have been called latent variables [9], i.e. variables that are hidden in the data and whose linear combinations account for the manifest variables that have been observed in order to construct the data matrix. The meaning of latent variables is explained in detail in Chapters 31 and 32 on the analysis of measurement tables and contingency tables. 

The number of singular vectors r is at most equal to the smallest of the number of rows n or the number of columns p of the data table X. For the sake of simplicity we will assume here that p is smaller than n, which is most often the case with measurement tables. Hence, we can state here that r is at most equal to p or equivalently that rindependent measurements in X. Independent measurements are those that cannot be expressed as a linear combination or weighted sum of the other variables. 

In the previous section we have developed principal components analysis (PCA) from the fundamental theorem of singular value decomposition (SVD). In particular we have shown by means of eq. (31.1) how an nxp rectangular data matrix X can be decomposed into an nxr orthonormal matrix of row-latent vectors U, a pxr orthonormal matrix of column-latent vectors V and an rxr diagonal matrix of latent values A. Now we focus on the geometrical interpretation of this algebraic decomposition. 

In CFA we can derive biplots for each of the three types of transformed contingency tables which we have discussed in Section 32.3 (i.e., by means of row-, column- and double-closure). These three transformations produce, respectively, the deviations (from expected values) of the row-closed profiles F, of the column-closed profiles G and of the double-closed data Z. It should be reminded that each of these transformations is associated with a different metric as defined by W and W. Because of this, the generalized singular vectors A and B will be different also. The usual latent vectors U, V and the matrix of singular values A, however, are identical in all three cases, as will be shown below. Note that the usual singular vectors U and V are extracted from the matrix. ... 

PLS has been introduced in the chemometrics literature as an algorithm with the claim that it finds simultaneously important and related components of X and of Y. Hence the alternative explanation of the acronym PLS Projection to Latent Structure. The PLS factors can loosely be seen as modified principal components. The deviation from the PCA factors is needed to improve the correlation at the cost of some decrease in the variance of the factors. The PLS algorithm effectively mixes two PCA computations, one for X and one for Y, using the NIPALS algorithm. It is assumed that X and Y have been column-centred as usual. The basic NIPALS algorithm can best be demonstrated as an easy way to calculate the singular vectors of a matrix, viz. via the simple iterative sequence (see Section 31.4.1) ... 

The quantities of are called the singular values of A and the columns of U and V are called the left and right singular vectors. If A is symmetric, positive-semidefinite, the eigenvalues and the singular values of A are equal if A is not symmetric, they are not. 

The matrix (I — AAO projects on the orthogonal complement of the column space of A or stated otherwise (I — AAO produces the residuals after projection onto the column space of A. Hence, components have to be found such that, after projection of X(C B) on these components, the residual variation is minimal in a least squares sense. This is what principal component analysis does (see Chapter 3) and a solution is to take the first P left singular vectors of X(C B). The components found are automatically in the column space of X(C B), and, therefore, in the column space of X. 

The unitary matrices U and V form orthonormal bases for the column (output) space and the row (input) space of G. The column vectors of u, denoted m, are called output singular vectors. The columns of V, denoted v are called input singular vectors. Because ... 

Singular value decomposition is now applied to all the Level 1 matrices. The product of the absolute value of the first column of the left singular vector of each of the SVD evaluations... 

Equation (8.90) is non-singular since it has a non-zero determinant. Also the two row and column vectors can be seen to be linearly independent, so it is of rank 2 and therefore the system is controllable. 

Equation (8.91) is singular since it has a zero determinant. Also the column vectors are linearly dependent since the second column is —5 times the first column and therefore the system is unobservable. 

Correspondence factor analysis can be described in three steps. First, one applies a transformation to the data which involves one of the three types of closure that have been described in the previous section. This step also defines two vectors of weight coefficients, one for each of the two dual spaces. The second step comprises a generalization of the usual singular value decomposition (SVD) or eigenvalue decomposition (EVD) to the case of weighted metrics. In the third and last step, one constructs a biplot for the geometrical representation of the rows and columns in a low-dimensional space of latent vectors. 

From the latent vectors and singular values one can compute the nxr generalized score matrix S and the pxr generalized loading matrix L. These matrices contain the coordinates of the rows and columns in the space spanned by the latent vectors ... 

Figure 2.1 Hyper-prism built on the column-vectors (alf a2, a3) of a non-singular matrix A3 x 3. The determinant is equal to the volume of the hyper-prism. When one of the vectors can be expressed as a linear combination of the others, both the volume and the determinant vanish and the matrix A is singular.

If det A =0, the column-vectors A are not linearly independent (nor are the column-rows) and the matrix is singular. At least one edge-vector of the hyper-prism made of the column-vectors is in the subspace of the remaining edge-vectors the volume of the hyper-prism vanishes and det A = 0. 

The first equation is the well-known Singular Value Decomposition. In the context of PCR the eigenvectors U form the basis for the column vectors of Y. The second equation in (5.72) attempts to also represent the column vector q of qualities in the same space U. If both representations are good then PCR works well, resulting in accurate predictions. A potential drawback of PCR is the fact that U is defined solely by Y. Even if there is good reasoning for a relationship between q and U, as indicated in the derivation of equation (5.60), it is somehow accidental. ... 

---