Principal component/factor analysis

The terms factor analysis, principal components analysis, and singular value decomposition (SVD) are used by spectroscopists to describe the fitting of a two-way array of data with a general bilinear model. We will use the term factor analysis in this sense, although this term has a somewhat different meaning in statistics. SVD is a specific algebraic procedure, discussed by Henry and Hofrichter and briefly later in this chapter, whose use alone is often not the best way to fit a general bilinear model. 

Prior to ANOVA, ratings for the 16 mood scales underwent factor analysis (principal components) of the correlation matrix and subsequent orthogonal rotation... 

Abdi, H., Williams, L. J. and Valentin, D. (2013). Mnltiple factor analysis principal component analysis for mnltitable and multiblock data sets. Wiley Interdisciplinary Reviews Computational Statistics, 5, 149-179. 

Multiple linear regression is strictly a parametric supervised learning technique. A parametric technique is one which assumes that the variables conform to some distribution (often the Gaussian distribution) the properties of the distribution are assumed in the underlying statistical method. A non-parametric technique does not rely upon the assumption of any particular distribution. A supervised learning method is one which uses information about the dependent variable to derive the model. An unsupervised learning method does not. Thus cluster analysis, principal components analysis and factor analysis are all examples of unsupervised learning techniques. 

Univac. Large scale systems, STAT-PACK, FACTAN - Factor and principal component analysis (1973) 33-39. 

The aim of factor analysis is to calculate a rotation matrix R which rotates the abstract factors (V) (principal components) into interpretable factors. The various algorithms for factor analysis differ in the criterion to calculate the rotation matrix R. Two classes of rotation methods can be distinguished (i) rotation procedures based on general criteria which are not specific for the domain of the data and (ii) rotation procedures which use specific properties of the factors (e.g. non-negativity). 

The goal of factor analysis (FA) and their essential variant principal component analysis (PCA) is to describe the structure of a data set by means of new uncorrelated variables, so-called common factors or principal components. These factors characterize frequently underlying real effects which can be interpreted in a meaningful way. 

An important application field of factor and principal component analysis is environmental analysis. Einax and Danzer [1989] used FA to characterize the emission sources of airborne particulates which have been sampled in urban screening networks in two cities and one single place. The result of factor analysis basing on the contents of 16 elements (Al, B, Ba, Cr, Cu, Fe, Mg, Mn, Mo, Ni, Pb, Si, Sn, Ti, V, Zn) determined by Optical Atomic Emission Spectrography can be seen in Fig. 8.17. In Table 8.3 the common factors, their essential loadings, and the sources derived from them are given. 

Although the software used was not a full-featured factor analysis program, portions of the printed output are useful in studying the spectral data set. Table VI shows some information obtainable from PCR models (large data set) with 5, 10 and 13 factors. In this case, the "factors" are principal components derived entirely from the sample data set. PLS factors are not interpretable in the same manner. 

As discussed in the introduction to this chapter, examining the row space of a matrix is an effective way of investigating the relationship between samples. However, this is only feasible when the number of measurement variables (columns) is less than three. Principal components analysis is a mathematical manipulation of a data matrix where the goal is to represent the variation present in many variables using a small number of "factors. A new row space is constructed in which to plot the samples by redefining the axes using factors rather than the original measurement variables. Tliesc new axes, referred to as factors or principal components (PCs), allow the analyst to probe matrices... 

The advantage of utilizing the standardized form of the variable is that quantities of different types can be included in the analysis including elemental concentrations, wind speed and direction, or particle size information. With the standardized variables, the analysis is examining the linear additivity of the variance rather than the additivity of the variable itself. The disadvantage is that the resolution is of the deviation from the mean value rather than the resolution of the variables themselves. There is, however, a method to be described later for performing the analysis so that equation 16 applies. Then, only variables that are linearly additive properties of the system can be included and other variables such as those noted above must be excluded. Equation 17 is the model for principal components analysis. The major difference between factor analysis and components analysis is the requirement that common factors have the significant values of a for more than one variable and an extra factor unique to the particular variable is added. The factor model can be rewritten as... 

Multivariate models have been successful in identifying source contributions in urban areas. They are not independent of Information on source composition since the chemical component associations they reveal must be verified by source emissions data. Linear regressions can produce the typical ratio of chemical components in a source but only under fairly restrictive conditions. Factor and principal components analysis require source composition vectors, though it is possible to refine these source composition estimates from the results of the analysis (6.17). 

B. Wold, Factor and Principal Component Analysis, in D.L. Massart,... 

The answers to these questions will usually be given by so-called unsupervised learning or unsupervised pattern recognition methods. These methods may also be called grouping methods or automatic classification methods because they search for classes of similar objects (see cluster analysis) or classes of similar features (see correlation analysis, principal components analysis, factor analysis). 

The X data matrix which contains all information describing the probe-target interactions can be analyzed by PCA [29, 30]. PCA is a multivariate projection method which allows one to extract the systematic information which is contained in the data matrix and to present it in a simplified form. The original number of variables is reduced to a few factors called principal components (PCs). The result of such an analysis can then be visualized by means of two informative plots which allow a straightforward interpretation of the problem. In this way, PCA provides an understanding of similarities and dissimilarities between the different protein binding sites with respect to their interaction with potential ligands. 

Another common method in factor analysis is principal component analysis. Principal component method can be considered a development of the previous procedure and commonly proceeds in a sequence of steps [74] as follows ... 

NIR spectroscopy became much more useful when the principle of multiple-wavelength spectroscopy was combined with the deconvolution methods of factor and principal component analysis. In typical applications, partial least squares regression is used to model the relation between composition and the NIR spectra of an appropriately chosen series of calibration samples, and an optimal model is ultimately chosen by a procedure of cross-testing. The performance of the optimal model is then evaluated using the normal analytical performance parameters of accuracy, precision, and linearity. Since its inception, NIR spectroscopy has been viewed primarily as a technique of quantitative analysis and has found major use in the determination of water in many pharmaceutical materials. 

S Wold. Cross-validatory estimation of the number of components in factor and principal components analysis. Technometrics, 20(4) 397-405, 1978. 

In this section we shall consider the rather general case where for a series of chemical compounds measurements are made in a number of parallel biological tests and where a set of descriptor variables is believed to be related to the biological potencies observed. In order to imderstand the data in their entirety and to deal adequately with the mathematical properties of such data, methods of multivariate statistics are required. A variety of such methods is available as, for example, multivariate regression, canonical correlation, principal component analysis, principal component regression, partial least squares analysis, and factor analysis, which have all been applied to biological or chemical problems (for reviews, see [1-11]). Which method to choose depends on the ultimate objective of an analysis and the property of the data. We have found principal component and factor analysis particularly useful. For this reason and also since many multivariate methods make use of components for factors we will start with these methods in some detail, while the discussion of other approaches will be less extensive. 

Specify the following terms in multivariate analysis principal component, eigenvector, common and unique factor, score, loading, target vector, latent variable. 

---