Scores and loadings

We have seen above that the r columns of U represent r orthonormal vectors in row-space 5 . Hence, the r columns of U can be regarded as a basis of an r-dimensional subspace 5 of 5 . Similarly, the r columns of V can be regarded as a basis of an r-dimensional subspace S of column-space 5. We will refer to S as the factor space which is embedded in the dual spaces S and SP. Note that r p n according to our convention. The concepts of row-, column-, and factor-spaces will be more fully developed in the next section. 

We define the coordinates for the n rows of X in factor space S by means of  

A traditional notation in chemometrics for SVD defines scores and loadings by means of the symbols T and P such that X = T P, which is equivalent to X = U A V, where T = U A and P = V. This notation corresponds with the case a = 1 and P = 0, which is the most frequently used combination of factor scaling coefficients in chemometrics. 

however, results in an abstract mathematical transformation of the original data matrix, which takes the form 

It is possible to calculate scores and loadings matrices as large as desired, provided that the common dimension is no larger than the smallest dimension of the original data matrix, and corresponds to the number of PCs that are calculated. 

Hence if the original data matrix is dimensions 30 x 28 (or / x. /), no more than 28 (nonzero) PCs can be calculated. If the number of PCs is denoted by A, then this number can be no larger than 28. 

Each scores matrix consists of a series of column vectors, and each loadings matrix a series of row vectors. Many authors denote these vectors by ta and pa, where a is the number of the principal component (1, 2, 3 up to A). The scores matrices T and P are composed of several such vectors, one for each principal component. The first 

The first three scores and loadings vectors for the data of Table 4.1 (case study 1) are presented in Table 4.3 for the first three PCs (A = 3). 

---

Fig. 31.2. Geometrical example of the duality of data space and the concept of a common factor space, (a) Representation of n rows (circles) of a data table X in a space Sf spanned by p columns. The pattern P" is shown in the form of an equiprobabi lity ellipse. The latent vectors V define the orientations of the principal axes of inertia of the row-pattern, (b) Representation of p columns (squares) of a data table X in a space y spanned by n rows. The pattern / is shown in the form of an equiprobability ellipse. The latent vectors U define the orientations of the principal axes of inertia of the column-pattern, (c) Result of rotation of the original column-space S toward the factor-space S spanned by r latent vectors. The original data table X is transformed into the score matrix S and the geometric representation is called a score plot, (d) Result of rotation of the original row-space S toward the factor-space S spanned by r latent vectors. The original data table X is transformed into the loading table L and the geometric representation is referred to as a loading plot, (e) Superposition of the score and loading plot into a biplot.

By virtue of the symmetry between scores and loadings, we can also construct bipolar axes through two columns 1 and 1 - such as is shown in Fig. 31.3f. When we project a row s, upon this bipolar axis we construct a difference between two elements in X. The proof follows readily from eq. (31.22) ... 

Biplots constructed from this table are shown in Figs. 31.5 to 31.11. The horizontal and vertical axes of these biplots represent scores and loadings of the... 

Generalized scores and loadings computed from Z in Table 32.6... 

Fig. 32.7. CFA biplot resulting from the superposition of the score and loading plots of Figs. 32.6a and b. The coordinates of the products and the disorders are contained in Table 32.9.

Figure 32.8 shows the biplot constructed from the first two columns of the scores matrix S and from the loadings matrix L (Table 32.11). This biplot corresponds with the exponents a = 1 and p = 1 in the definition of scores and loadings (eq. (39.41)). It is meant to reconstruct distances between rows and between columns. The rows and columns are represented by circles and squares respectively. Circles are connected in the order of the consecutive time intervals. The horizontal and vertical axes of this biplot are in the direction of the first and second latent vectors which account respectively for 86 and 13% of the interaction between rows and columns. Only 1% of the interaction is in the direction perpendicular to the plane of the plot. The origin of the frame of coordinates is indicated... 

In Chapter 31 we stated that any data matrix can be decomposed into a product of two other matrices, the score and loading matrix. In some instances another decomposition is possible, e.g. into a product of a concentration matrix and a spectrum matrix. These two matrices have a physical meaning. In this chapter we explain how a loading or a score matrix can be transformed into matrices to which a physical meaning can be attributed. We introduce the subject with an example from environmental chemistry and one from liquid chromatography. 

Suppose that 15 spectra have been measured at 20 wavelengths and that after normalization on a sum equal to one they are compiled in a data matrix. Suppose also that by PCA the following score and loading matrices are obtained ... 

Given a matrix of data PCA results in two quantities usually called scores and loadings. Scores are related to the measurements, and they... 

L contains normalised rows while T is weighted by the matrix S. This, however, is somewhat ambiguous as the decomposition of the transposed, Y1, is equally possible and then the score and loading matrices are simply exchanged. For this reason, we do not use the expressions scores and loadings. The Singular Value Decomposition maintains some kind of symmetry between the decompositions of Y and Yl. 

The described projection method with scores and loadings holds for all linear methods, such as PCA, LDA, and PLS. These methods are capable to compress many variables to a few ones and allow an insight into the data structure by two-dimensional scatter plots. Additional score plots (and corresponding loading plots) provide views from different, often orthogonal, directions. 

Factor analysis with the extraction of two factors and varimax rotation can be carried out in R as described below. The factor scores are estimated with a regression method. The resulting score and loading plots can be used as in PCA. 

Another possibility is the Tucker3 model where a decomposition of the array into sets of scores and loadings is performed that should describe the data in a more condensed form than the original data array. For the sake of simplicity we will describe the model for a three-way array, but it is easy to extend the idea to multiway data. Let xijk denote an element of a three-way array X of dimension I/J/K. Basic assumption is that the data are influenced by a relatively small set of driving forces (factors). Then the Tucker3 model is defined as... 

Thus for each mode a factorization (decomposition into scores and loadings) is done, expressed by three matrices and a three-way core-array G. The matrices A, B, C, and G are computed by minimizing the sum of squared errors the optimum number of factors, P, Q, R, can be estimated by cross validation. In a similar manner the Tucker2 model can be defined which reduces only two of the three modes, as well as the Tuckerl model which reduces only one of the three modes. 

PCA scores and loadings have unique properties as follows ... 

Earlier it was mentioned, and demonstrated using the Fisher Iris example (Section 12.2.5), that the PCA scores (T) can be used to assess relationships between samples in a data set. Similarly, the PCA loadings (P) can be used to assess relationships between variables in a data set. For PCA, the first score vector and the first loading vector make up the first principal component (PC), which represents the most dominant source of variability in the original x data. Subsequent pairs of scores and loadings ([score vector 2, loading vector 2], [score vector 3, loading vector 3]...) correspond to the next most dominant sources of variability. 

Figure 12.19 Scores and loadings obtained from PCA analysis of the NIR spectra of high-density/hw-denshy polyethylene blend films shown in Figure 12.18 (A) scatter plot of first 2 PC scores, (B) overlayed spectral plot of first two PC loadings. In the scores plot, different symbols are used to denote different blend compositions.

Earlier, it was mentioned that due to the orthogonality constraints of scores and loadings, as well as the variance-based criteria for their determination, it is rare that PCs and LVs obtained from a PC A or PLS model correspond to pure chemical or physical phenomena. However, if one can impose specihc constraints on the properties of the scores and or loadings, they can be rotated to a more physically meaningful form. The multivariate curve resolution (MCR) method attempts to do this for spectral data. 

Principal Component (PC) In this book, the tenn principal component is used as a generic term to indicate a factor or dimension when using SIMCA, principal components analysis, or principal components regression. Using this terminology, there are scores and loadings associated with a given PC. (See also Factor.)... 

Figure 4.2 Starting point original data and how a traditional PCA would decompose them. Since this is not an optimal solution, we need to define the scores and loadings in some other way that not only describe the main information present on X and Y, but at the same time relate them.

The most commonly used PCA algorithm involves sequential determination of each principal component (or each matched pair of score and loading vectors) via an iterative least squares process, followed by subtraction of that component s contribution to the data. Each sequential PC is determined such that it explains the most remaining variance in the X-data. This process continues until the number of PCs (A) equals the number of original variables (M), at which time 100% of the variance in the data is explained. However, data compression does not really occur unless the user chooses a number of PCs that is much lower than the number of original variables (A M). This necessarily involves ignoring a small fraction of the variation in the original X-data which is contained in the PCA model residual matrix E. 

---