Noise eigenvectors

Next, we find the first eigenvector of the noisy data set and plot it in Figures 41 and 42. We see that it is nearly identical to the first eigenvector of the noise... 

Continuing, we find the second eigenvector for the noisy data. Figures 43 and 44 contain plots of the first two eigenvectors for the noisy data. Again, the second eigenvector for the noisy data is nearly identical to that of the noise-free data. 

Finally, we calculate the eigenvalues for these eigenvectors. They are show in Table 7 together with the eigenvalues for the noise free data. 

Let s consider the nature of the variance spanned by the third eigenvectoi We know that it cannot contain any information that is related to th concentrations of the components in the samples because that information ca only lie in the plane of the original data. Thus, the Information-to-Noise ratio o the variance spanned by this eigenvector must be zero. 

So, we can discard the third eigenvector and, along with it, that portion of the variance in our spectra that displaced the data out of the plane of the noise-free data. We are in fact, discarding a portion of the noise without significantly distorting the spectra The portion of the noise we discard is called the extracted error or the residuals. Remember that the noise we added also displaced the points to some extent within the plane of the noise-free data. This portion of the noise remains in the data because it is spanned by the eigenvectors that we must retain. The noise that remains is called the imbedded error. The total error is sometimes called the real error. The relationship among the real error (RE), the extracted error (XE), and the imbedded error (IE) is... 

Since the noise is isotropic, each vector, whether a noise vector or a basis vector, picks up its equivalent share of the noise (we will see, soon, that we should take degrees-of-freedom into account when discussing what amount of noise is an equivalent share for each vector). If we had measured the spectra of our 2-component system at 100 wavelengths, we would, potentially be able to discard 98 out of a possible 100 eigenvectors. In doing so, we would expect to discard more noise than we can in this case. 

This process of discarding the noise eigenvectors to extract some of the noise from the data is sometimes called short circuit data reproduction. A more convenient term is regeneration. 

So now we understand that when we use eigenvectors to define an "abstract factor space that spans the data," we aren t changing the data at all, we are simply finding a more convenient coordinate system. We can then exploit the properties of eigenvectors both to remove noise from our data without significantly distorting it, and to compress the dimensionality of our data without compromising the information content. 

Factor spaces are a mystery no more We now understand that eigenvectors simply provide us with an optimal way to reduce the dimensionality of our spectra without degrading them. We ve seen that, in the process, our data are unchanged except for the beneficial removal of some noise. Now, we are ready to use this technique on our realistic simulated data. PCA will serve as a pre-processing step prior to ILS. The combination of Principal Component Analysis with ILS is called Principal Component Regression, or PCR. 

Thus, if we wish to compare the eigenvectors to one another, we can divide each one by equation [57] to normalize them. Malinowski named these normalized eigenvectors reduced eigenvectors, or REV". Figure 52 also contains a plot of the REV" for this isotropic data. We can see that they are all roughly equal to one another. If there had been actual information present along with the noise, the information content could not, itself, be isotropically distributed. (If the information were isotropically distributed, it would be, by definition, noise.) Thus, the information would be preferentially captured by the earliest... 

As we saw in the last chapter, by discarding the noise eigenvectors, we are able to remove a portion of the noise from our data. We have called the data that results after the noise removal the regenerated data. When we perform principal component regression, there is not really a separate, explicit data regeneration step. By operating with the new coordinate system, we are automatically regenerating the data without the noise. 

This lack of sharpness of the 1-way F-test on REV s is sometimes seen when there is information spanned by some eigenvectors that is at or below the level of the noise spanned by those eigenvectors. Our data sets are a good example of such data. Here we have a 4 component system that contains some nonlinearities. This means that, to span the information in our data, we should expect to need at least 4 eigenvectors — one for each of the components, plus at least one additional eigenvector to span the additional variance in the data caused by the non-linearity. But the F-test on the reduced eigenvalues only... 

All of the remaining factors do appear to contain nothing but noise. Remember that true noise eigenvectors will lie in some random direction that is devoid of any useful information. Thus, they should look like pure noise. 

Just as the spectral and concentration data points are exactly congruent with each other within the planes containing the data points, the spectral and concentration eigenvectors for this noise-free, perfectly linear case must also be exactly congruent. Because the vectors are congruent, the projection of each spectral data point onto a spectral factor must be directly proportional to the projection of the corresponding concentration data point onto the corresponding concentration factor ... 

When we calculate the eigenvectors for the two different data spaces (concentration and spectral spaces) we find the corresponding spectral and concentration vectors are shifted by different amounts in different directions. This is a consequence of the independence of the noises in the concentration and spectral spaces. So, just as the noise destroyed the perfect congruence between the noise-free spectral and concentration data points, it also destroyed... 

It is assumed that the structural eigenvectors explain successively less variance in the data. The error eigenvalues, however, when they account for random errors in the data, should be equal. In practice, one expects that the curve on the Scree-plot levels off at a point r when the structural information in the data is nearly exhausted. This point determines the number of structural eigenvectors. In Fig. 31.15 we present the Scree-plot for the 23x8 table of transformed chromatographic retention times. From the plot we observe that the residual variance levels off after the second eigenvector. Hence, we conclude from this evidence that the structural pattern in the data is two-dimensional and that the five residual dimensions contribute mostly noise. 

The two eigenvectors define a plane in the original variable space. This process can be repeated systematically until the eigenvalue associated with each new eigenvector is of such a small magnitude that it represents the noise associated with the observations more than it does information. In the limit where the number of significant eigenvectors equals the number... 

In a similar, slightly more complex, eye-based analysis, one can investigate the noisiness of the eigenvectors. The real eigenvectors or principal components are smooth. They have broad structures while noise eigenvectors are wildly oscillating and show no underlying structure. Of course, as before, the difference can be more or less pronounced. We analyse the same data as before ... 

Another interesting observation can be made the signs of the eigenvectors are not defined - they arbitrarily result from the Singular Value Decomposition. Apart from the amount of noise, the matrices Y and Y1 are identical, but the resulting eigenvectors have opposite signs. 

In this example, we are analysing the data set with the low noise level and, accordingly, the distinction between structured and noise residuals is crisp and unambiguous. Only noise is left after the subtraction of the contributions of three eigenvectors, equations (5.9) and (5.11). 

Recall, the standard deviation of the added noise in Y was lxlO-3. It is reached approximately after the removal of 3 sets of eigenvectors (at t=4). Note that, from a strictly statistical point of view, it is not quite appropriate to use Matlab s std function for the determination of the residual standard deviation since it doesn t properly take into account the gradual reduction in the degrees of freedom in the calculation of R. But it is not our intention to go into the depths of statistics here. For more rigorous statistical procedures to determine the number of significant factors, we refer to the relevant chemometrics literature on this topic. 

---