Principal components and factor analysis

Principal component analysis is based on the eigenvalue-eigenvector decomposition of the n h empirical covariance matrix Cy = X X (ref. 22-24). The eigenvalues are denoted by 2 — Vi where the last inequality follows from the presence of same random error in the data. Using the eigenvectors u, U2,. . ., un, define the new variables 

Another important problem is to reproduce the observation matrix using only the primary factors, i.e., dropping some small terms in (1.113) that likely stem from measurement error. 

Representing the data in terms of a small number of primary factors is a very efficient way of storing information. This approach is frequently used in spectroscopic libraries, designed to identify unknown species by comparing their spectra with ones filed in the library. 

You will better understand the goals of factor analysis considering first the highly idealized situation with error-free observations and only r n linearly independent columns in the matrix X. As discussed in Section 1.1, all columns of X are then in an r—dimensional subspace, and you can write them as linear combinations of r basis vectors. Since the matrix X X has now r nonzero eigenvalues, there are exactly r nonvanishing vectors in the matrix Z defined by (1.111), and these vectors form a basis for the subspace. The corresponding principal components z, z2,. .., zr are the coordinates in this basis. In the real life you have measurement errors, the columns of X 

Forsythe and C.B. Moler, Computer Solution of Linear Algebraic Systems, Prentice Hall, Englewood Cliffs,N.J., 1967. 

---

In the PARAF AC model, the three loading matrices A, B, C are not necessarily orthogonal [56]. The solution of the PARAF AC model, however, is unique and does not suffer from the indeterminacy that arises in principal components and factor analysis. 

CONTENTS 1. Chemometrics and the Analytical Process. 2. Precision and Accuracy. 3. Evaluation of Precision and Accuracy. Comparison of Two Procedures. 4. Evaluation of Sources of Variation in Data. Analysis of Variance. 5. Calibration. 6. Reliability and Drift. 7. Sensitivity and Limit of Detection. 8. Selectivity and Specificity. 9. Information. 10. Costs. 11. The Time Constant. 12. Signals and Data. 13. Regression Methods. 14. Correlation Methods. 15. Signal Processing. 16. Response Surfaces and Models. 17. Exploration of Response Surfaces. 18. Optimization of Analytical Chemical Methods. 19. Optimization of Chromatographic Methods. 20. The Multivariate Approach. 21. Principal Components and Factor Analysis. 22. Clustering Techniques. 23. Supervised Pattern Recognition. 24. Decisions in the Analytical Laboratory. 

For the descriptor matrix X used in principal components and factor analysis, the matrix X X is symmetric. Its eigenvalues are related to the sum of squares of the descriptors. The corresponding eigenvectors are the loading vectors of the principal component model. 

JE Jackson. Principal components and factor analysis Part I -principal components. J. Quality Technology, 12(4) 201-213, 1980. 

Anon., Principal Components and Factor Analysis , 2008, urkhttp // www.statsoft.com/textbook/stfacan.html, date accessed January 11, 2011. 

In this section we shall consider the rather general case where for a series of chemical compounds measurements are made in a number of parallel biological tests and where a set of descriptor variables is believed to be related to the biological potencies observed. In order to imderstand the data in their entirety and to deal adequately with the mathematical properties of such data, methods of multivariate statistics are required. A variety of such methods is available as, for example, multivariate regression, canonical correlation, principal component analysis, principal component regression, partial least squares analysis, and factor analysis, which have all been applied to biological or chemical problems (for reviews, see [1-11]). Which method to choose depends on the ultimate objective of an analysis and the property of the data. We have found principal component and factor analysis particularly useful. For this reason and also since many multivariate methods make use of components for factors we will start with these methods in some detail, while the discussion of other approaches will be less extensive. 

What is the difference between the principal component and factor analysis ... 

One possibility to speedup the search is preliminary sorting of the data sets. Here, the methods of unsupervised pattern recognition are used, for example, principal component and factor analysis, cluster analysis, or neural networks (cf. Sections 5.2 and 8.2). The unknown spectrum is then compared with every class separately. 

In the period 1970-2010, statistical methods, such as principal component and factor analysis, correlation, multiple regression and discriminate analysis, were... 

Both component and factor analysis as defined by equations 17 and 18 aim at the identification of the causes of variation in the system. The analyses are performed somewhat differently. For the principal components analysis, the matrix of correlations defined by equation 10 is used. For the factor analysis, the diagonal elements of the correlation matrix that normally would have a value of one are replaced by estimates of the amount of variance that is within the common factor space. This problem of separation of variance and estimation of the matrix elements is discussed by Hopke et al. (4). 

The diffusion of correlation methods and related software packages, such as partial-least-squares regression (PLS), canonical correlation on principal components, target factor analysis and non-linear PLS, will open up new horizons to food research. 

MULTIVARIATE PRINCIPAL COMPONENTS ANALYSIS OF AGING BEEF Multivariate principal component or factor analysis was performed on data obtained fi-om samples of aging beef (described above). Factor analysis was used since this method facilitates the visual examination of existing relationships (correlations) among the experimental treatments and the sensory, chemied. 

An alternative to the use of principal components or factor analysis is the BCUT method of Pearlman [Pearlman and Smith 1998]. In this method, three square matrices are constructed for each molecule. Each matrix is of a size equal to the number of atoms in the molecule and has as its elements various atomic and interatomic parameters. One matrix is intended to represent atomic charge properties, another represents atomic polarisabilities and the third hydrogen-bonding capabilities. These quantities can be computed with semi-empirical... 

Murray-Rust, P Motherwell, S. Computer retrieval and analysis of molecular geometry. 1. General principles and methods, Acta Cryst. 1978, B34, 2518-2526. A rough application of principal components, or factor analysis, is a shirt that has just one size parameter (S, M, F, XF), instead of a specification of waistline, chest width, arm, or leg lengths, etc., in the assumption that these body parameters are correlated. 

Liu, G., SwUiart, M.T., Neelamegham, S. Sensitivity, principal component and flux analysis applied to signal transduction the case of epidermal growth factor mediated signaling. Bioinformatics 21, 1194—1202 (2005)... 

Multiple linear regression is strictly a parametric supervised learning technique. A parametric technique is one which assumes that the variables conform to some distribution (often the Gaussian distribution) the properties of the distribution are assumed in the underlying statistical method. A non-parametric technique does not rely upon the assumption of any particular distribution. A supervised learning method is one which uses information about the dependent variable to derive the model. An unsupervised learning method does not. Thus cluster analysis, principal components analysis and factor analysis are all examples of unsupervised learning techniques. 

More detailed statistical analyses (chemical element balance, principal component analysis and factor analysis) demonstrate that soil contributes >50% to street dust, iron materials, concrete/cement and tire wear contribute 5-7% each, with smaller contributions from salt spray, de-icing salt and motor vehicle emissions (5,93-100). A list is given in Table VII of the main sources of the elements which contribute to street dust. 

The goal of factor analysis (FA) and their essential variant principal component analysis (PCA) is to describe the structure of a data set by means of new uncorrelated variables, so-called common factors or principal components. These factors characterize frequently underlying real effects which can be interpreted in a meaningful way. 

Principal Component Analysis (PCA) is the most popular technique of multivariate analysis used in environmental chemistry and toxicology [313-316]. Both PCA and factor analysis (FA) aim to reduce the dimensionality of a set of data but the approaches to do so are different for the two techniques. Each provides a different insight into the data structure, with PCA concentrating on explaining the diagonal elements of the covariance matrix, while FA the off-diagonal elements [313, 316-319]. Theoretically, PCA corresponds to a mathematical decomposition of the descriptor matrix,X, into means (xk), scores (fia), loadings (pak), and residuals (eik), which can be expressed as... 

More commonly, we are faced with the need for mathematical resolution of components, using their different patterns (or spectra) in the various dimensions. That is, literally, mathematical analysis must supplement the chemical or physical analysis. In this case, we very often initially lack sufficient model information for a rigorous analysis, and a number of methods have evolved to "explore the data", such as principal components and "self-modeling analysis (21), cross correlation (22). Fourier and discrete (Hadamard,. . . ) transforms (23) digital filtering (24), rank annihilation (25), factor analysis (26), and data matrix ratioing (27). 

S. Wold, Cross-validatory estimation of the number of components in factor analysis and principal component models. Technometrics, 20, 397-406 (1978). 

In this paper the PLS method was introduced as a new tool in calculating statistical receptor models. It was compared with the two most popular methods currently applied to aerosol data Chemical Mass Balance Model and Target Transformation Factor Analysis. The characteristics of the PLS solution were discussed and its advantages over the other methods were pointed out. PLS is especially useful, when both the predictor and response variables are measured with noise and there is high correlation in both blocks. It has been proved in several other chemical applications, that its performance is equal to or better than multiple, stepwise, principal component and ridge regression. Our goal was to create a basis for its environmental chemical application. 

Haaland and coworkers (5) discussed other problems with classical least-squares (CLS) and its performance relative to partial least-squares (PLS) and factor analysis (in the form of principal component regression). One of the disadvantages of CLS is that interferences from overlapping spectra are not handled well, and all the components in a sample must be included for a good analysis. For a material such as coal LTA, this is a significant limitation. 

In some diseases a simple ordinal scale or a VAS scale cannot describe the full spectrum of the disease. There are many examples of this including depression and erectile dysfunction. Measurement in such circumstances involves the use of multiple ordinal rating scales, often termed items. A patient is scored on each item and the summation of the scores on the individual items represents an overall assessment of the severity of the patient s disease status at the time of measurement. Considerable amoimts of work have to be done to ensure the vahdity of these complex scales, including investigations of their reprodu-cibihty and sensitivity to measuring treatment effects. It may also be important in international trials to assess to what extent there is cross-cultural imiformity in the use and imderstand-ing of the scales. Complex statistical techniques such as principal components analysis and factor analysis are used as part of this process and one of the issues that need to be addressed is whether the individual items should be given equal weighting. 

Wold, S., Cross-Validatory Estimation of the Number of Components in Factor Analysis and Principal Component Models Technometrics 1978, 20, 397-406. 

---