Analysis principal components

Principal component analysis (PCA) is aimed at explaining the covariance structure of multivariate data through a reduction of the whole data set to a smaller number of independent variables. We assume that an m-point sample is represented by the nxm matrix X which collects i=l.m observations (measurements) xt of a column-vector x with j=, .n elements (e.g., the measurements of n=10 oxide weight percents in m = 50 rocks). Let x be the mean vector and Sx the nxn covariance matrix of this sample 

Given the eigencomponent decomposition of the symmetric covariance matrix 

The n coordinates associated with the n eigenvectors define the vector e, of the point vector x( in the new system of coordinates, which is written formally as 

Applying equation (4.3.4) for variable change, we get the component covariance matrix Se as 

Since A is diagonal, the covariance of the components along the jth eigenvector is 

The scope of Principal Component Analysis (PCA) is a consistent portrayal of a data set in a representation space. Mathematically, PCA is a linear transformation that may be described as S=WX. Here X is the original data set, W is the transformation matrix, and S are the data in the representation space. PCA is the simplest and most widely used method of multivariate analysis. Nonetheless, most users are seldom aware of its assumptions and sometimes results are badly interpreted. 

The peculiarity of PCA is in a representation of the data set onto a subspace of reduced dimensionality where the statistical properties of the original data set are preserved. 

Although the PCA concept is used in many disciplines it was strongly developed in chemometrics, where it was introduced at the beginning to analyze spectroscopic and chromatographic data, which are characterized by a higher correlation among the spectra channels [8]. 

The possibility of a reliable representation of a chemical sensor array data set in subspaces of smaller dimension lies in the fact that the individual sensors always exhibit a high correlation among themselves. PCA consists of finding an orthogonal basis where the correlation among sensors disappears. 

As a consequence, in a sensor space of dimension N the effective dimension of the sub-space occupied by the data is less than N. This dimension can be precisely evaluated using algorithms developed to describe dynamic systems. An example is the correlation distance that allows evaluating the fractional dimensionality of a data-set [16]. Correlation distance provides an independent way to evaluate the expected reduction of dimension. 

From a statistical perspective, principal component analysis (PCA) is a method for reducing the dimensionality of data sets by transforming correlated variables into a smaller set of uncorrelated variables andfinding linear combinations of the original variables with maximum variability. 

As chemical processes become increasingly instrumented and data is recorded more frequently, both data compression and extraction become important. However, the information should be compressed in such a way that the essential features of the data are safeguarded and the new data is more easily displayed than the original data. 

PCA is a tool, initially used by chemometricians for data compression and extraction of analytical instruments. Nowadays, it is a tool that is also frequently used to analyze data from process plants, where process data is often correlated. PCA offers the advantage that it finds a set of new uncorrelated variables that are a linear combination of the original variables. The new variables describe the maximum variance in the original data set in ascending order. 

Principal component analysis (PCA) can be perfonned on data matrices, consisting of raw data. Analysis of raw data can be used to estimate the importance and characteristics of the individual variables. The data matrix for n variables measured by m data points is given by  

PCA entails transforming n original variables xj, X2,. .., x into an ordered sequence of n 

The simplest and most widely used chemometric technique is Principal Component Analysis (PCA). Its objective is to accomplish orthogonal projection and in that process identify the minimum number of sensors yielding the maximum amount of information. It removes redundancies from the data and therefore can be called a true data reduction tool. In the PCA terminology, the eigenvectors have the meaning of Principal Components (PC) and the most influential values of the principal component are called primary components. Another term is the loading of a variable i with respect to a PQ. 

High loading of PCf indicates that this vector is aligned close to the original data values that is, the transformation to the feature space defined by the new principal components matches the important (perhaps the most important) trend in the raw data. Conversely, low loading means that the PC does not match a significant trend in the data. Typically, 2-3 PCs can characterize most experimental datasets. This then allows a 2-D or a 3-D graphical representation of the results, as shown in Fig. 10.6. As powerful as it is, the PCA fails in cases where the individual sensors in the 

The eigenvectors and eigenvalues of C are then calculated. Calling V the eigenvector matrix and D the diagonal matrix of the eigenvalues, the composition of the principal components is given as a matrix, F, of linear combination coefficients that mix the 

Eigenvalues that are significantly different from zero identify the relevant components the percent contribution of each principal component k to the effective variance of the sample can be determined as follows  

Both in bivariate and multivariate analysis, correlations between molecular and crystal properties are poor, unsure, and scarcely robust. The fundamental reason is that crystal packing is a matter of finely adjusted detail, and global indicators will never reach the subtleness required for a real predictive power. 

Both CLS and ILS have strengths and weaknesses, and it would be beneficial to combine their strengths and eliminate the weaknesses. A commonly accepted procedure to achieve this goal is to use spectral decomposition in the form of principal component analysis (PCA). 

The advantages of CLS and ILS can be combined if we can reduce the number of wavenumbers at which absorbances are measured for ILS, yet retain the advantages of the use of the entire spectrum as in CLS regression. To do this we must consider what a collection of calibration spectra represents. First, we have the pure components. The calibration spectra are composed of different concentrations of die pure components, or for the spectra, different scalings. Unfortunately, there is more that we must consider. If the components interact, we have interaction spectra, and these will be present in different ratios in the individual calibration mixture spectra. Each instrument will bias the spectra, so we have instrument-induced artifacts that may vary with temperamre, total absorbance, wavenumber, and so on. In all we have numerous contributions or factors that together comprise each of the individual calibration spectra. The only difference between all the individual calibration spectra is the proportion of the contributions. 

If we had an ideal spectrometer and absolutely no interaction effects, we would be able to construct the individual calibration spectra from the spectra of the pure components alone. More important, we should be able to decompose the individual calibration spectra into the pure component spectra. Because we do not have ideal spectrometers and rarely have ideal calibration sets, this is not possible. Nonetheless, if it were possible, we could simply represent each spectram in the calibration set as the proportions of the pure component spectra. For example, if there were three components, each calibration spectrum would be composed of the three components in different proportions. Three numbers would represent the proportions, one for each component. 

The problem we have with ILS is that we cannot use as many wavenumbers as we can in CLS. In CLS, this was a distinct advantage, as the use of all the absorbances led to a more accmate prediction. If in the ILS model we use the proportions, or scores, in place of the absorbances, we have restricted the dependent variables to a small number as required in ILS we also have a score for each component, also as required in ILS. On the other hand, the scores represent the entire absorbance spectram and by default we have incorporated all the absorbance information and its associated advantage. 

The problem lies in how we can extract pure components from a real data set. In reality such an operation is difficult, although several algorithms exist that can do precisely this. These algorithms are beyond the scope of this treatise, and recovery of the pure components spectra is neither necessary nor often desirable. It is more important to decompose a set of calibration spectra into significant factors. These significant factors are also called principal components, eigenvectors, loadings. 

In addition, since we have more than one variable, it is possible to calculate a product-moment (Pearson) correlation coefficient for each pair of variables. These are summarized in the correlation matrix in Table 8.2, obtained using Minitab. 

One problem with multivariate data is that its sheer volume may make it difficult to see patterns and relationships. For example, a spectrum would normally be characterized by several hundred intensity measurements rather than just four as in Table 8.1 and in this case the correlation matrix would contain hundreds of values. Thus the aim of many methods of multivariate analysis is data reduction. Quite frequently there is some correlation between the variables, as there is for the data in Table 8.1, and so some of the information is redundant. Principal component analysis (PCA) is a technique for reducing the amount of data when there is correlation present. It is worth stressing that it is not a useful technique if the variables are uncorrelated. 

The idea behind PCA is to iind principal components Zi, Z2. Z which are linear combinations of the originai variables describing each specimen, X, X2. X , i.e. 

The principal components are obtained from the covariance matrix. The term covariance (see Section 5.3) is a measure of the joint variance of two variables. The covariance matrix for the data in Table 8.1 is 

Carry out a principal component analysis of the data in Table 8.1. 

The samples are grouped so that those with the most similar odour character are closest together on the map (e.g. Lilial and Bourgeonal ), and those that are most different are furthest apart (e.g, cyclamen aldehyde and Lyral ). The arrows indicate the direction of increasing perception of the odour characteristics shown. Only the odour characteristics that significantly correlate with the distribution of the samples across the map are shown these are the characteristics that are responsible for the systematic differences between the samples. 

All the materials were perceived to be floral and muguet in character, so these characteristics are not shown on the map. Cyclamen aldehyde, Lilial and Bourgeonal are the most fruity of the samples and are grouped together on the right-hand side of the map, while Lyral and hydroxycitronellal are the sweetest materials and Mayol is the most herbal. 

This involves describing linear interrelationships on the basis of correlation calculations and co-variance calculations. Groups are formed such that within each group the variables belonging thereto are highly correlated, but between the groups of individual variables there is no linear interrelationship. 

The diagram depicts the correlation of variables with different principal components (factors), expressed as principal component load. As has already 

---

The essential degrees of freedom are found by a principal component analysis of the position correlation matrix Cy of the cartesian coordinate displacements Xi with respect to their averages xi), as gathered during a long MD run ... 

The important underlying components of protein motion during a simulation can be extracted by a Principal Component Analysis (PGA). It stands for a diagonalization of the variance-covariance matrix R of the mass-weighted internal displacements during a molecular dynamics simulation. 

Grubmiiller described a method to induce conformational transitions in proteins and derived rate constants for these ([Grubmiiller 1994]). The method employs subsequent modifications of the original potential function based on a principal component analysis of a short MD simulation. It is discussed in more detail in the chapter of Eichinger et al. in this volume. 

Hayward et al. 1994] Hayward, S., Kitao, A., Go, N. Harmonic and anharmonic aspects in the dynamics of BPTI A normal mode analysis and principal component analysis. Prot. Sci. 3 (1994) 936-943 [Head-Gordon and Brooks 1991] Head-Gordon, T., Brooks, C.L. Virtual rigid body dynamics. Biopol. 31 (1991) 77-100... 

Step 2 This ensemble is subjected to a principal component analysis (PCA) [61] by diagonalizing the covariance matrix C G x 7Z, ... 

Steven Hayward, Akio Kitao, and Nobuhiro Go. Harmonic and anharmonic aspects in the dynamics of BPTI A normal mode analysis and principal component analysis. Physica Scripta, 3 936-943, 1994. 

M. A. Balsera, W. Wriggers, Y. Oono, and K. Schulten. Principal component analysis and long time protein dynamics. J. Phys. Chem., 100 2567-2572, 1996. 

We have to apply projection techniques which allow us to plot the hyperspaces onto two- or three-dimensional space. Principal Component Analysis (PCA) is a method that is fit for performing this task it is described in Section 9.4.4. PCA operates with latent variables, which are linear combinations of the original variables. 

To gain insight into chemometric methods such as correlation analysis, Multiple Linear Regression Analysis, Principal Component Analysis, Principal Component Regression, and Partial Least Squares regression/Projection to Latent Structures... 

Kohonen network Conceptual clustering Principal Component Analysis (PCA) Decision trees Partial Least Squares (PLS) Multiple Linear Regression (MLR) Counter-propagation networks Back-propagation networks Genetic algorithms (GA)... 

PCR is a combination of PCA and MLR, which are described in Sections 9.4.4 and 9.4.3 respectively. First, a principal component analysis is carried out which yields a loading matrix P and a scores matrix T as described in Section 9.4.4. For the ensuing MLR only PCA scores are used for modeling Y The PCA scores are inherently imcorrelated, so they can be employed directly for MLR. A more detailed description of PCR is given in Ref. [5. ... 

I Principal Component Analysis (PCA) transforms a number of correlated variables into a smaller number of uncorrelated variables, the so-called principal components. 

The previously mentioned data set with a total of 115 compounds has already been studied by other statistical methods such as Principal Component Analysis (PCA), Linear Discriminant Analysis, and the Partial Least Squares (PLS) method [39]. Thus, the choice and selection of descriptors has already been accomplished. 

Spectral features and their corresponding molecular descriptors are then applied to mathematical techniques of multivariate data analysis, such as principal component analysis (PCA) for exploratory data analysis or multivariate classification for the development of spectral classifiers [84-87]. Principal component analysis results in a scatter plot that exhibits spectra-structure relationships by clustering similarities in spectral and/or structural features [88, 89]. 

The dimensionality of a data set is the number of variables that are used to describe eac object. For example, a conformation of a cyclohexane ring might be described in terms c the six torsion angles in the ring. However, it is often found that there are significai correlations between these variables. Under such circumstances, a cluster analysis is ofte facilitated by reducing the dimensionality of a data set to eliminate these correlation Principal components analysis (PCA) is a commonly used method for reducing the dimensior ality of a data set. 

An alternative to principal components analysis is factor analysis. This is a technique which can identify multicollinearities in the set - these are descriptors which are correlated with a linear combination of two or more other descriptors. Factor analysis is related to (and... 

Multiple linear regression is strictly a parametric supervised learning technique. A parametric technique is one which assumes that the variables conform to some distribution (often the Gaussian distribution) the properties of the distribution are assumed in the underlying statistical method. A non-parametric technique does not rely upon the assumption of any particular distribution. A supervised learning method is one which uses information about the dependent variable to derive the model. An unsupervised learning method does not. Thus cluster analysis, principal components analysis and factor analysis are all examples of unsupervised learning techniques. 

The field points must then be fitted to predict the activity. There are generally far more field points than known compound activities to be fitted. The least-squares algorithms used in QSAR studies do not function for such an underdetermined system. A partial least squares (PLS) algorithm is used for this type of fitting. This method starts with matrices of field data and activity data. These matrices are then used to derive two new matrices containing a description of the system and the residual noise in the data. Earlier studies used a similar technique, called principal component analysis (PCA). PLS is generally considered to be superior. 

Other chemometrics methods to improve caUbration have been advanced. The method of partial least squares has been usehil in multicomponent cahbration (48—51). In this approach the concentrations are related to latent variables in the block of observed instmment responses. Thus PLS regression can solve the colinearity problem and provide all of the advantages discussed earlier. Principal components analysis coupled with multiple regression, often called Principal Component Regression (PCR), is another cahbration approach that has been compared and contrasted to PLS (52—54). Cahbration problems can also be approached using the Kalman filter as discussed (43). 

A method of resolution that makes a very few a priori assumptions is based on principal components analysis. The various forms of this approach are based on the self-modeling curve resolution developed in 1971 (55). The method requites a data matrix comprised of spectroscopic scans obtained from a two-component system in which the concentrations of the components are varying over the sample set. Such a data matrix could be obtained, for example, from a chromatographic analysis where spectroscopic scans are obtained at several points in time as an overlapped peak elutes from the column. 

Principal component analysis has been used in combination with spectroscopy in other types of multicomponent analyses. For example, compatible and incompatible blends of polyphenzlene oxides and polystyrene were distinguished using Fourier-transform-infrared spectra (59). Raman spectra of sulfuric acid/water mixtures were used in conjunction with principal component analysis to identify different ions, compositions, and hydrates (60). The identity and number of species present in binary and tertiary mixtures of polycycHc aromatic hydrocarbons were deterrnined using fluorescence spectra (61). 

How does principal component analysis work Consider, for example, the two-dimensional distribution of points shown in Figure 7a. This distribution clearly has a strong linear component and is closer to a one-dimensional distribution than to a full two-dimensional distribution. However, from the one-dimensional projections of this distribution on the two orthogonal axes X and Y you would not know that. In fact, you would probably conclude, based only on these projections, that the data points are homogeneously distributed in two dimensions. A simple axes rotation is all it takes to reveal that the data points... 

---