Description of Factor Analysis

The reader has seen in Section 5.4.1 that the total number of eigenvalues and eigenvectors, which is the same as the number of features, reproduces the correlation matrix and therefore describes the total variance of the data. The general model of both PCA and FA described in Section 5.4.1 is therefore called complete factor solution. 

When analyzing real data sets one has to find common factor structures which explain the main part of the variance of the data. Therefore in factor analysis the total variance of the data is divided by the reduced factor solution into the three parts  

The common feature variance originates from correlating features. Specific feature variance and residuals or error are now expressed by the matrix E  

X - data matrix A - factor loadings F - factor scores E - residuals 

The communality is introduced as a mathematical measure of this common feature variance. The communality is the part of the variance of one feature which is described by the common factor solution in the factor analysis. High communalities, hj, mean that this feature variance is highly explained by the factor solution. Low communalities for one feature detect either a specific feature variance or high random error. 

---

Mathematical description of Factor Analysis and Principal Components Analysis... 

---