Variance explained

The portion of the total variance associated with they-th characteristic root or eigenvalue is  

As can be seen in this example, one principal component captures more than 40% of all of the variance in the data setX. 

The first score vector 6 (or first principal component) can in this case be written as  

Note MATLAB computes -P rather than P, therefore the signs in Eqn (22.12) are opposite the signs in the P matrix from the eigs computation). 

It can be seen that Eqn. (22.12) is a static model representation. PCA is useful for systems with many (correlated) variables. If a process has 40 measured variables, it is often possible to define a few principal components that capture most of the variance in the originally measured variables, thereby achieving a considerable system dimensionality reduction. 

---

As in example 1, the explained variance (the total variance minus the residual variance) is calculated by comparing the true process data with estimates computed from a reference model. This explained variance can be computed as a function of the batch number, time, or variable number. A large explained variance indicates that the variability in the data is captured by the reference model and that correlations exist among the variables. The explained variance as a function of time can be very useful in differentiating among phenomena that occur in different stages of the process operations. 

Figure 38 shows the variance explained by the two principal component (PC) model as a percentage of each of the two indices batch number and time. The lower set of bars in Fig. 38a are the explained variances for the first PC, while the upper set of bars reflects the additional contribution of the second PC. The lower line in Fig. 38b is the explained variance over time for the first PC and the upper line is the combination of PC 1 and 2. Figure 38a indicates, for example, that batch numbers 13 and 30 have very small explained variances, while batch numbers 12 and 33 have variances that are captured very well by the reference model after two PCs. It is impossible to conclude from this plot alone, however, that batches 13 and 30 are poorly represented by the reference model. 

Additional insight is possible from Fig. 38b. Here we see that the magnitude of the explained variance accounted for by the second PC has noticeably increased after minute 70. This is consistent because, from process knowledge, it is known that removal of water is the primary event in the first part of the batch cycle, while polymerization dominates in the later part, explaining why the variance profile changes around the 70-minute point. 

Fig. 38. Explained variance by batches (a) and over time (b) for batch polymer reactor data.

The overall interpretation comes together when we recall that batches 13 and 30 had small explained variances. We note that the (7-statistic for batch 13 indicates that it is within the 95% limit for both PCs while the Q-statistic of batch 30 is not. The conclusion is that the variations in batch 13 are small random deviations about the average batch. In the case of batch 30, larger variations occur that are not well explained by the reference model. These variations are either large random fluctuations or variations that are orthogonal to the model subspace. Hence, the quality of batch 30, with a high probability, will not be within the specified limits. 

A variety of statistical parameters have been reported in the QSAR literature to reflect the quality of the model. These measures give indications about how well the model fits existing data, i.e., they measure the explained variance of the target parameter y in the biological data. Some of the most common measures of regression are root mean squares error (rmse), standard error of estimates (s), and coefficient of determination (R2). 

Fig. 5 Main contamination sources identified by PCA for sediments, fish, and suface water in the Ebro River basin, and explained variances for each principal component. Variable identification. Organic compounds in sediments 1, summatory of hexachlorocyclohexanes (HCHs) 2, summa-tory of DDTs (DDTs) 3, hexachlorobenzene (HCB) 4, hexachlorobutadiene (HCBu) 5, summatory of trichlorobenzenes (TCBs) 6, naphthalene 7, fluoranthene 8, benzo(a)pyrene 9, benzo(b) fluoranthene 10, benzo(g,h,i)perylene 11, benzo(k)fluoranthene 12, indene(l,2,3-cd)pyrene. Organic compounds in fish 1, hexachlorobenzene (HCB) 2, summatory of hexachlorocyclohexanes (HCHs) 3, o,p-DDD 4, o,p-DDE 5, o,p-DDT 6, p,p-DDD 7, />,/>DDE 8, />,/>DDT 9, summatory of DDTs (DDTs) 10, summatory of trichlorobenzenes (TCBs) 11, hexachlorobutadiene (HCBu) 12, fish length. Physico-chemical parameters in water 1, alkalinity 2, chlorides 3, cyanides 4, total coliforms 5, conductivity at 20°C 6, biological oxygen demand 7, chemical oxygen demand 8, fluorides 9, suspended matter 10, total ammonium 11, nitrates 12, dissolved oxygen 13, phosphates 14, sulfates 15, water temperature 16, air temperature...

Table 4.13. Eigenvalues and percentage of explained variance for the oceanic island isotope...

There are several important issues for PCA, like the explained variances of each PC which determine the number of components to select. Moreover, it is of interest if outliers have influenced the PCA calculation, and how well the objects are presented in the PCA space. These and several other questions will be treated below. 

Finally, a measure of lack of fit using a PCs can be defined using the sum of the squared errors (SSE) from the test set, flSSETEST = Latest 2 (prediction sum of squares). Here, 2 stands for the sum of squared matrix elements. This measure can be related to the overall sum of squares of the data from the test set, SStest = -Xtest 2- The quotient of both measures is between 0 and 1. Subtraction from 1 gives a measure of the quality of fit or explained variance for a fixed number of a PCs ... 

This measure can be related to the sum of squared elements of the columns of X to obtain a proportion of unexplained variance for each variable. Subtraction from 1 results in a measure Qj of explained variance for each variable using a PCs... 

As an example, we consider the glass vessels data and apply PCA using the classical estimators. The left plot in Figure 3.14 shows the values i Qj using one PC to fit the data. The quality of fit is very low for S03, K20, and PbO. In the right plot two PCs are used and the measures 2Q] are shown in barplots. Except for SO3 the explained variances increased essentially. 

FIGURE 3.14 Explained variance for each variable using one (left) and two (right) PCs. The data used are the glass vessels data from Section 1.5.3. 

Tab. 9.4 Rat versus human bioactivity data comparison using entries from WOMBAT.2004.1 N is the number of compounds, R is the correlation coefficient, and is the fraction of explained variance...

It was mentioned earlier that PCA is a useful method for compressing the information contained in a large number of x variables into a smaller number of orthogonal principal components that explain most of the variance in the x data. This particular compression method was considered to be one of the foundations of chemometrics, because many commonly used chemometric tools are also focused on explaining variance and dealing with colinearity. However, there are other compression methods that operate quite differently than PCA, and these can be useful as both compression methods and preprocessing methods. 

For the styrene-butadiene example, the use of the PCR method to develop a calibration for di-butadiene is summarized in Table 12.6. It should be mentioned that the data were mean-centered before application of the PCR method. Figure 12.12 shows the percentage of explained variance in both x (the spectral data) andy (the c/i-butadiene concentration data) after each principal component. After four principal components, it does not appear that the use of any additional PCs results in a large increase in the explained variance of X or y. If a PCR regression model using four PCs is built and applied to the calibration data, a fit RMSEE of 1.26 is obtained. 

Figure 12.12 The percentage of explained variance in both the x data (solid line) and y data (dotted line), as a function of the number of PCs in the PCR regression model for c s-butadiene content in styrene-butadiene copolymers).

The difference between PLS and PCR is the manner in which the x data are compressed. Unlike the PCR method, where x data compression is done solely on the basis of explained variance in X followed by subsequent regression of the compressed variables (PCs) to y (a simple two-step process), PLS data compression is done such that the most variance in both x and y is explained. Because the compressed variables obtained in PLS are different from those obtained in PCA and PCR, they are not principal components (or PCs) Instead, they are often referred to as latent variables (or LVs). 

In the styrene-bntadiene copolymer example. Figure 12.13 shows the explained variance in both x and y as a function of the nnmber of PLS latent variables. When this explained variance graph is compared to the... 

The principle of PCA consists of finding the directions in space—known as principal components (PCs)—along which the data points are furthest apart. It requires linear combinations of the initial variables that contribute most to making the samples different from each other. PCs are computed iteratively, with the first PC carrying the most information, that is, the most explained variance, and the second PC carrying most of the residual information not taken into account by the previous PC, and so on. This process can go on until as many PCs have been computed as there are potential variables in the data table. At that point, all between-sample variation has been accounted for, and the PCs form a new set of axes having two... 

Total explained variance measures how much of the original variation in the data is described by the model. It expresses the proportion of structure found in the data by the model. Total residual and explained variances show how well the model fits... 

The problem is complex (as is any trade-off balance) and it can be approached graphically by the well-known Taguchi s loss function, widely used in quality control, slightly modified to account for Faber s discussions [30]. Thus, Figure 4.14 shows that, in general, the overall error decreases sharply when the first factors are introduced into the model. At this point, the lower the number of factors, the larger is the bias and the lower the explained variance. When more factors are included in the model, more spectral variance is used to relate the spectra to the concentration of the standards. Accordingly, the bias decreases but, at the same time, the variance in the predictions... 

Principal Components Analysis (PCA) Compression by explained variance... 

Table 8.3 The explained variance in the X-data, for each PC, for the iris data set...

---