Residual data matrix

A crucial operation in the NIPALS algorithm is the calculation of the residual data matrix which is independent of the contributions by the first singular vector. This can be produced by the instruction ... 

Subtract the effect of the new PC from the data matrix to get a residual data matrix... 

If it is desired to compute further PCs, substitute the residual data matrix for X and go to step 2. 

Dominant correlations of data are usually captured by a small number of initial eigenvectors. A simple orthogonal decomposition is accomplished by partitioning U = [UmU/j] and A = [Am A ] where M designates the number of initial dominant modes to be used for approximation while R stands for the remaining N — M) modes or the residual. Data matrix becomes Y = JM- M + R- R = Ym + Yr. For a successful approximation, YM captures significant variability trends and Yr simply represents residual random noise. Transformation in the form Y Ym uses M N + K) data entries and provides [1 — M N + K)/ NK)]100 % data compression. 

In order to apply residual bilinearization [35] at least two data sets are needed X which is the data set measured for the unknown sample and X which is the data matrix of a calibration standard, containing the analyte of interest. In the absence of interferences these two data matrices are related to each other as follows ... 

The residuals can be calculated from a given set of calibration samples in a different way. Cross validation is an important procedure to estimate a realistic prediction error like PRESS. The data for k samples are removed from the data matrix and then predicted by the model. The residual errors of prediction of cross-validation in this case are given by... 

If, however, the standard deviations, ayij, for all elements of the matrix Y are known or can be estimated reliably, it does make sense to use this information in the data analysis. Then, instead of the sum of squares, it is the sum of all appropriately weighted and squared residuals that has to be minimised. This is known as the chi-square or x2 -fitting. If the data matrix Y has the dimensions nsxnl, %2 is defined by... 

This empirical statistical function, based on the residual standard deviation (RSD), reaches a minimum when the correct number of factors are chosen. It allows one to reduce the number of columns of R from L to K eigenvectors or pure components. These K independent and orthogonal eigenvectors are sufficient to reproduce the original data matrix. As they are the result of a mathematical treatment of matrices, they have no physical meaning. A transformation (i.e. a rotation of the eigenvectors space) is required to find other equivalent eigenvectors which correspond to pure components. 

Fig. 8.1. A key idea in chemometrics is to record K variables (including spectral data) for S samples to form a matrix S X K. The critical characteristics of both the samples and the spectral data can then be understood using a smaller set of matrices S x M and M x K while the unmodeled residual remains available for analysis as the matrix e. While the reduced matrices provide insight into the system or process, residual data can be used to understand errors or limitations of the model...

A linear combination of different factors in the matrix A with factor scores in the matrix F can reproduce the data matrix X. These factors are new synthetic variables and represent a certain quantity of features of the data set. They explain the total variance of all features in a descending order and are themselves noncorrelated. It is, therefore, possible to reduce the dimension m of the data set with a minimum loss of information expressed by the matrix of residuals E. 

X - data matrix A - factor loadings F - factor scores E - residuals... 

It can be shown that the scores for the first eigenvector and principal component extract the maximum possible amount of variance from the original data matrix using a linear factor [15]. In other words, the first principal component is a least-squares result that minimizes the residual matrix. The second principal component extracts the maximum amount of variance from whatever is left in the first residual matrix. 

Because experimental error is always present in a measured data matrix, the corresponding row-mode eigenvectors (or eigenspectra) form an orthonormal set of basis vectors that approximately span the row space of the original data set. Figure 4.14 illustrates this concept. The distance between the endpoints of a and a is equal to the variance in a not explained by x and y, that is, the residual variance. 

The expected residual class variance for class q is calculated by using the residual data vectors for all samples in the training set. The resulting residual matrix is used to calculate the residual variance within class q. This value is an indication of how tight a class cluster is in multidimensional space. It is calculated according to Equation 4.46, where s02 is the residual variance in class q and n is the number of samples in class q. 

In this equation, daugv is the concentration of SVOC j in sample i in the augmented experimental data matrix IP1 8. The variable um im (the score of component n on row i) is the contribution of contamination source n in sample i. The variable vjn (the loading of variable j on component n) is the contribution of SVOC j in contamination source n. The residual, eip is the variance in sample i and variable j of not modeled by the N environmental contamination sources. The same equation can be written in matrix form as ... 

The 3D HNCO data matrix is very sparsely populated with crosspeaks because there is only one correlation per residue in the protein (Fig. 12.59). The main purpose of this experiment is to count residues and make sure all of the peaks can be found and identified. Once we have the assignments for each H-N pair, the data can be arranged in a strip plot in order of residue number. 

What are the residual sum of squares for the V and V blocks as each successive component is computed (hint start from the centred data matrix and simply sum the squares of each block, repeat for the residuals) What percentage of the overall variance is accounted for by each component ... 

Briefly, PCA summarizes the variation of a data matrix as a product of two lower-dimensional matrices, the score matrix Tand the loadings matrix P. With n objects and k grid points, the (n x k) data matrix is decomposed into the n x a) score matrix and the (a x k) loadings matrix plus a (n x k) error matrix of residuals E ... 

The second phase of this sensitivity study requires the baseline variability in samples from residue free animals be known. This data will establish the negative control and will also indicate if components from a residue free matrix will interfere in the assay. 

---