Mathematical Description of PCA

The first step is to compute the averages of each descriptor variable. This yields the average vector, x = [xtx2. .. xk] which gives the average matrix, X, after multiplication by the (nx 1) vector 1 = [11. .. 1].  

The matrix X can then be written as a sum of the average matrix and a residual matrix E. The matrix E describes the variation around the mean, it is this variation which is described by the principal components. 

The variances and covariance of the descriptors are given by the matrix (X — X) (X — X), in which the diagonal elements are the variances of the variables and the off-diagonal elements are the covariances. When the data have been scaled to unit variance, this matrix is called the correlation matrix and the off-diagonal elements are correlation coefficients for the correlations between the variables, and the sum of the variances is equal to the number of variables. 

As (X — X) (X — X) is a real symmetric matrix, it can be factorized into a diagonal matrix of eigenvalues, L, and matrices of the corresponding eigenvectors, P and P, cf canonical analysis (see Sect. 3.7) 

The eigenvectors of (X — X) (X — X) are the principal component vectors Pi and the eigenvalues X describe how much of the total variance is accounted for by Pi. 

---