Scree plots

This empirical test is based on the so-called Scree-plot which represents the residual variance as a function of the number of eigenvectors that have been extracted [42]. The residual variance V of the r -th eigenvector is defined by  

It is assumed that the structural eigenvectors explain successively less variance in the data. The error eigenvalues, however, when they account for random errors in the data, should be equal. In practice, one expects that the curve on the Scree-plot levels off at a point r when the structural information in the data is nearly exhausted. This point determines the number of structural eigenvectors. In Fig. 31.15 we present the Scree-plot for the 23x8 table of transformed chromatographic retention times. From the plot we observe that the residual variance levels off after the second eigenvector. Hence, we conclude from this evidence that the structural pattern in the data is two-dimensional and that the five residual dimensions contribute mostly noise. 

---

Fig. 31.15. Scree-plot, representing the residual variance V as a function of the number of factors r that has been extracted. The diagram is based on a factor analysis of Table 31.2 after log double-centering. A break point occurs after the second factor, which suggests the presence of only two structural factors, the residual factors being attributed to noise and artefacts in the data.

FIGURE 3.5 Scree plot for an artificial data set with eight variables, v, variance of PCA scores (percent of total variance) v climul. cumulative variance of PCA scores. 

PCA components with small variances may only reflect noise in the data. Such a plot looks like the profile of a mountain after a steep slope a more flat region appears that is built by fallen, deposited stones (called scree). Therefore, this plot is often named scree plot so to say, it is investigated from the top until the debris is reached. However, the decrease of the variances has not always a clear cutoff, and selection of the optimum number of components may be somewhat subjective. Instead of variances, some authors plot the eigenvalues this comes from PCA calculations by computing the eigenvectors of the covariance matrix of X note, these eigenvalues are identical with the score variances. 

If PCA is used for dimension reduction and creation of uncorrelated variables, the optimum number of components is crucial. This value can be estimated from a scree plot showing the accumulated variance of the scores as a function of the number of used components. More laborious but safer methods use cross validation or bootstrap techniques. 

As mentioned, hierarchical cluster analysis usually offers a series of possible cluster solutions which differ in the number of clusters. A measure of the total within-groups variance can then be utilized to decide the probable number of clusters. The procedure is very similar to that described in Section 5.4 under the name scree plot. If one plots the variance sum for each cluster solution against the number of clusters in the respective solution a decay pattern (curve) will result, hopefully tailing in a plateau level this indicates that further increasing the number of clusters in a solution will have no effect. 

Fig. 5-19. Scree plot of seven eigenvalues from the interlaboratory comparison...

The basis of the application of FA was a data matrix containing 17 features of 52 sedimented airborne particulate samples collected during a period when buildings were being heated. According to Fig. 7-10 the application of the scree plot [CATTELL, 1966] indicates four common factors. 

Fig. 7-10. Scree plot for the determination of the number of common factors...

To choose the optimal number of loadings fc there are many criteria. For a detailed overview, see Joliffe [56], A very popular graphical one is based on the scree plot, which exposes the eigenvalues in decreasing order. The index of the last component before the plot flattens is then selected. 

According to the eigenvalue results present in Table 6(b), and displayed in the scree plot of Figure 13, over 84% of the total variance in the original data... 

Figure 13 An eigenvalue, scree plot for the heart-tissue trace metal data...

Flgiue 15 A scree plot for the eigenvalues derivedfrom the IR spectra of 21 polymers... 

Figure 19 The scree plot of the eigenvalues extracted from the MS data...

Sampling theory, 27 Savitsky-Golay coefficients, 41 Savitsky-Golay differentiation, 57 Savitsky-Golay smoothing, 38 Scatter plot, 24 Scores, factor, 74 Scree plot, 75... 

In the following, the so-called scree plot for determining the number of components in PCA will be described. Afterwards, it will be shown how this method can be modified for finding the appropriate number of components in PARAFAC and in Tucker3 models. 

A certain cutoff in the scree plot is used to determine which components are too small to be used. For exploratory purposes, it may suffice to choose a cutoff value that leads to, e.g., 80 % of the variation explained for noisy data, but for more quantitative purposes it is useful to have a more elaborate determination of the appropriate cutoff value. Usually the number of components is chosen where the plot levels off to a linear decreasing pattern (see also Horn [1965]). Thus, no more than the number of factors to the left of this point should be retained. 

A plot similar to a scree plot can be made for PARAFAC, by plotting the sum of squares of the individual components. However, in this case, cumulative plots cannot be made directly because the variances of the individual factors are not additive due to the obliqueness of the factors. Furthermore, the sum of squares of the one-component model may not equal the size of the largest component in a two-component model. Hence, the scree plot is not directly useful for PARAFAC models. The cumulative scree plot for PARAFAC models, on the other hand, can be constructed by plotting the explained or residual sum of squares for a one-component model, a two-component model, etc. This will provide similar information to the ordinary two-way cumulative scree plot, with the exception that the factors change for every model, since PARAFAC is not sequentially fit. The basic principle is retained though, as the appropriate number of components to use is chosen as the number of components for which the decrease in the residual variation levels off to a linear trend (see Example 7.3). 

Using scree plots for determining dimensionality of PARAFAC model... 

Figure 7.5. The scree plot shows percentage of the total sum of squares explained by the PARAFAC model with increasing number of components. Results from fit and (expectation maximization) cross-validated residuals are shown.

As in ordinary two-way analysis, the scree plot based on fit values can sometimes be misleading, if the model is overfitting. Exchanging fit residuals with cross-validated... 

The scree plots described above are also used in regression models of the type y = Xb + ey where y is decomposed into a model Xb and a residual ey. Plots of the percentage variation explained as a function of the number of PCA orPLS components are used to study the fit of the regression model. With results from cross-validation or a test-set, a similar plot can be used to select the rank of the regression model. 

Specific cross-validation schemes for three-way data are given main emphasis. The choice of models and model hierarchy are explained. It is important to get a good fit and parsimony. The selection of appropriate model rank by the use scree plots, residual analysis and split-half analysis is introduced. Different ways of calculating residual statistics and leverages for three-way arrays are presented. 

The order of the topics treated is basically as plots would be used in an ongoing analysis scree plots, line plots, scatter plots, special plots as an aid in understanding the model, residual plots, leverage plots. 

---