Column space

In the illustration of Fig. 29.4 we regard the matrix X as either built up from n horizontal rows of dimension p, or as built up from p vertical columns x,.of dimension n. This exemplifies the duality of the interpretation of a matrix [9]. From a geometrical point of view, and according to the concept of duality, we can interpret a matrix with n rows and p columns either as a pattern of n points in a p-dimensional space, or as a pattern of p points in an n-dimensional space. The former defines a row-pattern P" in column-space 5, while the latter defines a column-pattern P in row-space S". The two patterns and spaces are called dual (or conjugate). The term dual space also possesses a specific meaning in another... 

Fig. 29.5. Geometrical interpretation of an nxp matrix X as either a row-pattern of n points P" in p-dimensional column-space S (left panel) or as a column-pattern of p points / in n-dimensional row-space S" (right panel). The p vectors Uy form a basis of 5 and the n vectors v, form a basis of 5".

Thus we have illustrated that the number of independent rows, the number of independent columns and the rank of the matrix are all identical. Hence, from geometrical considerations, we conclude that the ranks of the patterns in row- and column-space must also be equal. The above illustration is also rendered geometrically in Fig. 29.7. 

Fig. 29.7. Illustration of a pattern of points with rank of 2. The pattern is represented by a matrix X with dimensions 5x4 and a linear dependence between the three columns of X is assumed. The rank is shown to be the smallest number of dimensions required to represent the pattern in column-space 5 and in row-space S".

Fig. 29.8. (a) Pattern of points in column-space S (left panel) and in row-space S" (right panel) before column-centering, (b) After column-centering, the pattern in 5 is translated such that the centroid coincides with the origin of space. Distances between points in S are conserved while those in S" are not. (c) After column-standardization, distances between points in S and 5" are changed. Points in 5" are located on a (hyper)sphere centered around the origin of space. 

We have seen above that the r columns of U represent r orthonormal vectors in row-space 5". Hence, the r columns of U can be regarded as a basis of an r-dimensional subspace 5 of 5". Similarly, the r columns of V can be regarded as a basis of an r-dimensional subspace S of column-space 5. We will refer to S as the factor space which is embedded in the dual spaces S" and SP. Note that r

factor-spaces will be more fully developed in the next section. 

The columns of the loading matrix L contain the principal components of X in column-space S . 

In Chapter 29 we introduced the concept of the two dual data spaces. Each of the n rows of the data table X can be represented as a point in the p-dimensional column-space S . In Fig. 31.2a we have represented the n rows of X by means of the row-pattern F. The curved contour represents an equiprobability envelope, e.g. a curve that encloses 99% of the points. In the case of multinormally distributed data this envelope takes the form of an ellipsoid. For convenience we have only represented two of the p dimensions of SP which is in reality a multidimensional space rather than a two-dimensional one. One must also imagine the equiprobability envelope as an ellipsoidal (hyper)surface rather than the elliptical curve in the figure. The assumption that the data are distributed in a multinormal way is seldom fulfilled in practice, and the patterns of points often possess more complex structure than is shown in our illustrations. In Fig. 31.2a the centroid or center of mass of the pattern of points appears at the origin of the space, but in the general case this needs not to be so. 

In Fig. 31.2a we have represented the ith row x, of the data table X as a point of the row-pattern F in column-space S . The additional axes v, and V2 correspond with the columns of V which are the column-latent vectors of X. They define the orientation of the latent vectors in column-space S. In the case of a symmetrical pattern such as in Fig. 31.2, one can interpret the latent vectors as the axes of symmetry or principal axes of the elliptic equiprobability envelopes. In the special case of multinormally distributed data, Vj and V2 appear as the major and minor... 

Fig. 31.2. Geometrical example of the duality of data space and the concept of a common factor space, (a) Representation of n rows (circles) of a data table X in a space Sf spanned by p columns. The pattern P" is shown in the form of an equiprobabi lity ellipse. The latent vectors V define the orientations of the principal axes of inertia of the row-pattern, (b) Representation of p columns (squares) of a data table X in a space y spanned by n rows. The pattern / is shown in the form of an equiprobability ellipse. The latent vectors U define the orientations of the principal axes of inertia of the column-pattern, (c) Result of rotation of the original column-space S toward the factor-space S spanned by r latent vectors. The original data table X is transformed into the score matrix S and the geometric representation is called a score plot, (d) Result of rotation of the original row-space S toward the factor-space S spanned by r latent vectors. The original data table X is transformed into the loading table L and the geometric representation is referred to as a loading plot, (e) Superposition of the score and loading plot into a biplot.

The vector of column-means nip defines the coordinates of the centroid (or center of mass) of the row-pattern P" that represents the rows in column-space Sf . Similarly, the vector of row-means m defines the coordinates of the center of mass of the column-pattern that represents the columns in row-space S". If the column-means are zero, then the centroid will coincide with the origin of SP and the data are said to be column-centered. If both row- and column-means are zero then the centroids are coincident with the origin of both 5" and S . In this case, the data are double-centered (i.e. centered with respect to both rows and columns). In this chapter we assume that all points possess unit mass (or weight), although one can extend the definitions to variable masses as is explained in Chapter 32. 

Geometrically, column-centering of X is equivalent to a translation of the origin of column-space toward the centroid of the points which represent the rows of the data table X. Hence, the operation of column-centering leaves distances between the row-points unchanged. 

This corresponds with a choice of factor scaling coefficients a = 1 and p = 0, as defined in Section 31.1.4. Note that classical PCA implicitly assumes a Euclidean metric as defined above. Let us consider the yth coordinate axis of column-space, which is defined by a p-vector of unit length of the form ... 

In Section 29.3 it has been shown that a matrix generates two dual spaces a row-space S" in which the p columns of the matrix are represented as a pattern P , and a column-space S in which the n rows are represented as a pattern P". Separate weighted metrics for row-space and column-space can be defined by the corresponding metric matrices and W. This results into the complementary weighted spaces and S, each of which can be represented by stretched coordinate axes using the stretching factors in -J v and, where the vectors w and Wp contain the main diagonal elements of W and W. ... 

In the previous section we have seen that axes defined by the column variables can be rotated. It is also possible to rotate the principal components. Instead of rotating the axes which define the column space of X, we rotate here the significant PCs in the sub-space defined by V ... 

The pure variable technique can be applied in the column space (wavelength) as well as in the row space (time). When applied in the column space, a pure column is one of the column factors. In LC-DAD this is the elution profile of the compound which contains that selective wavelength in its spectrum. When applied in the row space, a pure row is a pure spectrum measured in a zone where only one compound elutes. 

Let us begin by representing a row matrix M = (1,2,3) in column space as shown in Figure 14-1. Note that the row vector M = (1,2, 3) projects onto the plane defined by columns 1 and 2 as a point (1, 2) or a vector (straight line) with a Cx direction angle (a) equal to... 

Figure 14-1 A representation of a row vector M = [1,2, 3] in column space, and the projection of this vector onto the plane represented by Columns 1 and 2.

The space of the column-vectors x such as Ax=Om< where 0m is a m x m matrix of zeroes, is called the nullspace of the matrix A. Any vector from the nullspace is therefore orthogonal to any vector from the row-space. The left nullspace of A is the set of vectors ym such as yTA = 0 . Any vector from the left nullspace is therefore orthogonal to any vector from the column-space. The left nullspace of A is identical to the nullspace of AT. 

The rank r of the matrix A is the dimension of its column-space, i.e., the maximum number of independent column-vectors. It can be demonstrated that the ranks of A and AT are equal. The dimension of the nullspace is n — r, that of the left nullspace m—r. 

Each vector x can be decomposed as the sum of a vector from the row-space and a vector in the nullspace. These two vectors are orthogonal. Each vector ym can be decomposed as the sum of a vector from the column-space and a vector in the left nullspace. These two vectors are orthogonal. 

Given the m x n rectangular matrix A, the m x m projector P=A(ATA) lAT projects each m-vector y onto the column-space of A. Py can be written... 

In the virtual mineral space, the rock composition is projected onto the plane made by the vectors enstatite [0,1,0]T and diopside [0,0,1]T. Although these vectors are not orthogonal in the original oxide composition space, which can be verified by constructing the dot product of columns 2 and 3 in the matrix BT, the particular choice of the projection makes the vectors orthogonal in the transformed space. According to the projector theory developed above, we project the rock composition onto the column-space of the matrix A such that... 

Figure 5.1 The least-square estimate y of the solution to equation (5.1.4) is the orthogonal projection of the observation vector y onto the column-space 1 a2, of matrix A.

The top of the column is frequently covered with a circle of filter paper or a layer of clean sand to prevent disturbance of the surface during subsequent loading. A suitably concentrated solution of the mixture is added from a pipette, the liquid is allowed to drain just to the surface of the adsorbent and the inside of the tube is rinsed with a small quantity of the solvent which is again allowed to drain just on to the column. Finally the column space above the adsorbent is filled with solvent and a dropping funnel filled with solvent is attached. 

---