Singular value decomposition principal component analysis

A whole spectrum of statistical techniques have been applied to the analysis of DNA microarray data [26-28]. These include clustering analysis (hierarchical, K-means, self-organizing maps), dimension reduction (singular value decomposition, principal component analysis, multidimensional scaling, or correspondence analysis), and supervised classification (support vector machines, artificial neural networks, discriminant methods, or between-group analysis) methods. More recently, a number of Bayesian and other probabilistic approaches have been employed in the analysis of DNA microarray data [11], Generally, the first phase of microarray data analysis is exploratory data analysis. 

In the previous section we have developed principal components analysis (PCA) from the fundamental theorem of singular value decomposition (SVD). In particular we have shown by means of eq. (31.1) how an nxp rectangular data matrix X can be decomposed into an nxr orthonormal matrix of row-latent vectors U, a pxr orthonormal matrix of column-latent vectors V and an rxr diagonal matrix of latent values A. Now we focus on the geometrical interpretation of this algebraic decomposition. 

We now have the data necessary to calculate the singular value decomposition (SVD) for matrix A. The operation performed in SVD is sometimes referred to as eigenanal-ysis, principal components analysis, or factor analysis. If we perform SVD on the A matrix, the result is three matrices, termed the left singular values (LSV) matrix or the V matrix the singular values matrix (SVM) or the S matrix and the right singular values matrix (RSV) or the V matrix. 

Principal Components Analysis (PCA) is a multivariable statistical technique that can extract the strong correlations of a data set through a set of empirical orthogonal functions. Its historic origins may be traced back to the works of Beltrami in Italy (1873) and Jordan in Prance (1874) who independently formulated the singular value decomposition (SVD) of a square matrix. However, the first practical application of PCA may be attributed to Pearson s work in biology [226] following which it became a standard multivariate statistical technique [3, 121, 126, 128]. 

Wall, M. E., Rechtsteiner, A., and Rocha, L. M. 2003. Singular value decomposition and principal component analysis. In A Practical Approach to Microarray Data Analysis, (eds. D. R Berrar, W. Dubitzky, M. Granzow), pp. 91-109, Norwell, MA Kluwer. 

A compromise was developed by Gabriel in 1971 and called the biplot [Gabriel 1971], This is also described by Jackson [1991] and Brereton [1992], It is useful to start with the simple case of two-way analysis. Principal component analysis of X is given as a singular value decomposition. A model with two principal components is (see Chapter 3) ... 

The terms factor analysis, principal components analysis, and singular value decomposition (SVD) are used by spectroscopists to describe the fitting of a two-way array of data with a general bilinear model. We will use the term factor analysis in this sense, although this term has a somewhat different meaning in statistics. SVD is a specific algebraic procedure, discussed by Henry and Hofrichter and briefly later in this chapter, whose use alone is often not the best way to fit a general bilinear model. 

The singular value decomposition (SVD) method, and the similar principal component analysis method, are powerful computational tools for parametric sensitivity analysis of the collective effects of a group of model parameters on a group of simulated properties. The SVD method is based on an elegant theorem of linear algebra. The theorem states that one can represent an w X n matrix M by a product of three matrices ... 

Principal components analysis, factor analysis, singular value decomposition, etc. are all techniques used in data reduction. The aim of the overall process is to reduce the data set X into the product of two matrices, T and P, with residual error matrix E ... 

Figure 4.4 Principal component analysis of a 6x3 matrix (a) the six samples in the original space of three measured variables, (b) the new axes (principal components PCi and PC2) obtained from the singular-value decomposition (SVD) of the 6x3 matrix and (c) representation of the six samples in the space of the principal components. Notice how the three original variables are correlated (the higher xi and X2 are, the higher is X3). Notice also how using only the coordinates (scores) of the samples on these two principal components, the relative position of the samples in the initial variable space is captured. This is possible because the original variables are correlated. Principal component regression (PCR) uses the scores on these two new variables (the two principal components) instead of the three originally measured variables.

Other powerful mathematical procedures to refine parameters not well defined by observations are the highly similar techniques of principal component analysis (PCA) and singular value decomposition (SVD). SVD can be written in the form of a set of slightly inconsistent linear equations, called condition equations ... 

---