Three-way principal components analysis

Multiway and particularly three-way analysis of data has become an important subject in chemometrics. This is the result of the development of hyphenated detection methods (such as in combined chromatography-spectrometry) and yields three-way data structures the ways of which are defined by samples, retention times and wavelengths. In multivariate process analysis, three-way data are obtained from various batches, quality measures and times of observation [55]. In image analysis, the three modes are formed by the horizontal and vertical coordinates of the pixels within a frame and the successive frames that have been recorded. In this rapidly developing field one already finds an extensive body of literature and only a brief outline can be given here. For a more comprehensive reading and a discussion of practical applications we refer to the reviews by Geladi [56], Smilde [57] and Henrion [58]. 

In the present terminology, a three-way data array is defined by n rows, p columns and q layers, with indices i,j and k, respectively. 

---

Barbieri, P., Andersson, C.A., Massart, D.L., Predozani, S., Adami, G., and Reisen-hofer, E., Modeling bio-geochemical interactions in the surface waters of the Gulf of Trieste by three-way principal component analysis (PCA), Anal. Chim. Acta, 398, 227-235, 1999. 

Marengo, E., Leardi, R., Robotti, E., Righetti, P.G., Antonucci, F. and Cecconi, D. (2003) Application of three-way principal component analysis to the evaluation of two-dimensional maps in proteomics./. Proteome Res., 2, 351-360. 

Henrion R, Body diagonalization of core matrices in three-way principal components analysis theoretical bounds and simulation, Journal of Chemometrics, 1993, 7, 477-494. 

Ten Berge JMF, Kiers HAF, Simplicity of core arrays in three-way principal component analysis and the typical rank of P x Q x 2 arrays, Linear Algebra and its Applications, 1999, 294, 169-179. 

V. Pravdova, C. Boucon, S. de Jong, B. Walczak, and D. L. Massart, Three-way principal component analysis applied to food analysis An example. Anal. Chim. Acta, 462 (2002) 133-148. 

Cocchi M, Durante C, Grandi M, Manzini D, Marchetti A. Three-way principal component analysis of the volatile fraction by HS-SPME/GC of aceto balsamico tradizionale of Modena. Talanta 2008 74 547-54. 

The reason for the correlation between the localization and the amino acid composition was sought by Andrade et al. (1998). They examined the amino acid composition of proteins with known localization and three-dimensional structure in three ways total composition, surface composition, and interior composition. The principal component analysis showed the best correlation between the surface composition and the localization. Therefore, they concluded that the correlation is the result of evolutionary adaptation of proteins to the surrounding environment. 

As discussed in the introduction to this chapter, examining the row space of a matrix is an effective way of investigating the relationship between samples. However, this is only feasible when the number of measurement variables (columns) is less than three. Principal components analysis is a mathematical manipulation of a data matrix where the goal is to represent the variation present in many variables using a small number of "factors. A new row space is constructed in which to plot the samples by redefining the axes using factors rather than the original measurement variables. Tliesc new axes, referred to as factors or principal components (PCs), allow the analyst to probe matrices... 

In the preceding description of the Mahalanobis distance, the number of coordinates in the distance metric is equal to the number of spectral frequencies. As discussed earlier in the section on principal component analysis, the intensities at many frequencies are dependent, and by using the full spectrum, we fit the noise in addition to the real information. In recent years, Mahalanobis distance has been defined with PCA or PLS scores instead of the spectral frequencies because these techniques eliminate or at least reduce most of the overfitting problem. The overall application of the Mahalanobis distance metric is the same except that the rt intensity values are replaced by the scores from PCA or PLS. An example of a Mahalanobis distance calculation on a set of Raman spectra for 25 carbohydrates is shown in Fig. 5-11. The 25 spectra were first subjected to PCA, and it was found that the first three principal components could account for most of the variance in the spectra. It was first assumed that all 25 spectra belonged to the same class because they were all carbohydrates. However, as shown in the three-dimensional plot in Fig. 5-11, the spectra can be clearly divided into three separate classes, with two of the spectra almost equal distance from each of the three classes. Most of the components in the upper left class in the two-dimensional plot were sugars however, some sugars were found in the other two classes. For unknowns, scores have to be calculated from the principal components and processed in the same way as the spectral intensities. 

Among the multivariate methods the most important are principal components analysis (PCA), factor analysis, cluster analysis and the pattern recognition method, from which only PCA will be briefly described below. PCA is used to find such a system of new variables, called principal components (PC), which explains the variation of a given data set in a more convenient way than the original system of variables, e.g. xl9...,Xj,...,xm. The greater convenience of PC consists mainly in a reduction of dimensions, m, in which the data were originally described, because the PC variables are chosen so that only two or three of them should be sufficient to characterize the variation of the data. The PC are linear combinations of the original variables, xj9 used to characterize the set of objects,... 

Principal component analysis enables reduction of a large data matrix into two or three main components that include orthogonally relevant information. In such a way, changes in metabolic profiles, described by many variables, can be measurably determined and compared. Subsequently, using other calculation procedures for the reduced data matrix, the importance of variables (metabolites) can be determined and assessed. Discrimination or regression calculation methods are of great importance in this step of the analysis. 

Statistical analyses. Three-way analyses of variance treating judges as a random effect were performed on each descriptive term using SAS Institute Inc. IMP 3.1 (Cary, North Carolina). Principal component analysis of the correlation matrix of the mean intensity ratings was performed with Varimax rotation. Over 200 GC peaks... 

A short introduction to principal component analysis for two-way arrays and PARAFAC for three-way arrays will be given here. These methods are used in the next chapter, and therefore, a short introduction is necessary. Definitions, notations and other details on these methods are explained in later chapters. 

In principal component analysis (PCA), a matrix is decomposed as a sum of vector products, as shown in Figure 1.6. The vertical vectors (following the object way) are called scores and the horizontal vectors (following the variable way) are called loadings. A similar decomposition is given for three-way arrays. Here, the array is decomposed as a sum of triple products of vectors as in Figure 1.7. This is the PARAFAC model. The vectors, of which there are three different types, are called loadings. 

Clearly, these residuals are far from perfect, and thus, even though the data are known to be perfectly trilinear with two components, the sequential PARAFAC algorithm fails to find a reasonable estimate of the parameters. However, this difference between sequential and simultaneous fitting is not related to the three-way nature of the PARAFAC model. Rather it is the orthogonality of the components in principal component analysis that enables the components to be calculated sequentially. A simple two-way example will help in illustrating this. 

Ledyard Tucker was one of the pioneers in multi-way analysis. He proposed [Tucker 1964, Tucker 1966] a series of models nowadays called A-mode principal component analysis or Tucker models. An extensive treatment of Tucker models is given by Kroonenberg and de Leeuw [1980] and Kroonenberg [1983], In the following, three different Tucker models will be treated. 

Especially scatter plots should be interpreted very carefully. Principal component analysis produces latent variables and at the same time orthogonal scores and orthonormal loadings. The latent variable and the Euclidean interpretation of scatter plots both come from the same model. For three-way models, the latent variables do not allow a direct Euclidean interpretation of loading plots. A recalculation can give this Euclidean interpretation, but then the original latent variable structure gets lost. 

In excited-state spectroscopies, including fluorescence spectroscopy, spectroscopic intensity is usually linear in functions of each of three or more independent variables, so that a three-way array of data can be fit with a trilinear model. The presence of three or more linear relationships makes algebraic methods for resolving the spectra and other properties of individual components substantially more powerful than in the case of two linear relationships. The use of a general trilinear model is sometimes known as three-way factor analysis, three-mode factor analysis, or threemode principal component analysis. For a review of the mathematics and application to spectroscopy, see our survey article. ... 

The basic idea of PARAFAC is so flexible and general that its ap>plication is not limited to a particular field of spectroscopy confined to a sp>ecific electromagnetic probe. Examples of the application include fluorescence (Christensen et at, 2005 Rinnan et al., 2005), IR (Wu et ah, 2003), NMR (Bro et ah, 2010), UV (Ebrahimi et al., 2008 Van Benthem et al., 2011) and mass spectroscopy (Amigo et ah, 2008). The first p>art of this chapter covers the theoretical background of trilinear decomposition of three-way data by PARAFAC with comparison to bilinear decomposition of two-way data by Principal component analysis (PC A). 

During 1983-1986 winter, 71 natural stands scattered in the Aquitaine forest area (Fig. 2) were analysed. Many of them were severely frost damaged. For each stand, the analysis of a mean sample was compared to standards by the way of a principal components analysis. The three principal components PC + PC2 +... 

If the origin ( 0 ) is chosen at the centroid of the atoms, then it can be shown that distances from this point can be computed from the interatomic distances alone. A fundamental theorem of distance geometry states that a set of distances can correspond to a three-dimensional object only if the metric matrix g is rank three, i.e., if it has toee positive and N — 3 zero eigenvalues. This is not a trivial theorem, but it may be made plausible by thinking of the eigenanalysis as a principal component analysis all of the distance properties of the molecule should be describable in terms of three components , which would be the x, y and z coordinates. If we denote the eigenvector matrix as w and the eigenvalues A., the metric matrix can be written in two ways ... 

---