Calculating the Covariance Matrix

We will retain the calculations for all covariances in a symmetric m by m covariance matrix S with the entry at row i and column j containing Si j while the main diagonal contains the variance for row(0. 

Select the eigenvector with the highest eigenvalue to define the principle component of the data set, that is, the one showing the most significant relationship between the data dimensions. 

Sort the eigenvectors in order of decreasing eigenvalues, which gives the components in order of significance. 

Optionally, ignore the components of lesser significance. This will reduce information content the higher the eigenvalues are. However, it will reduce the final data set to fewer dimensions than the original in fact, the number of dimensions is reduced by the number of eigenvectors left out. 

Derive the new data set by taking the transpose of the vector and multiply it on the left of the original data set, transposed  

---

The performances of the indirect conventional methods described previously are very sensitive to outliers, so they are not robust. The main reason for this is that they use a direct method to calculate the covariance matrix of the residuals (). If outliers are present in the sampling data, the assumption about the error distribution will be... 

Principal component analysis (PCA) takes the m-coordinate vectors q associated with the conformation sample and calculates the square m X m matrix, reflecting the relationships between the coordinates. This matrix, also known as the covariance matrix C, is defined as... 

Essentially this is equivalent to using (Sf/dk kj instead of (<3f/<3k,) for the sensitivity coefficients. By this transformation the sensitivity coefficients are normalized with respect to the parameters and hence, the covariance matrix calculated using Equation 12.4 yields the standard deviation of each parameter as a percentage of its current value. 

The elements of the covariance matrix of the parameter estimates are calculated when the minimization algorithm has converged with a zero value for the Marquardt s directional parameter. The covariance matrix of the parameters COV(k) is determined using Equation 11.1 where the degrees of freedom used... 

As already discussed in Chapter 11, matrix A calculated during each iteration of the Gauss-Newton method can be used to determine the covariance matrix of the estimated parameters, which in turn provides a measure of the accuracy of the parameter estimates (Tan and Kalogerakis, 1992). 

Statistical properties of a data set can be preserved only if the statistical distribution of the data is assumed. PCA assumes the multivariate data are described by a Gaussian distribution, and then PCA is calculated considering only the second moment of the probability distribution of the data (covariance matrix). Indeed, for normally distributed data the covariance matrix (XTX) completely describes the data, once they are zero-centered. From a geometric point of view, any covariance matrix, since it is a symmetric matrix, is associated with a hyper-ellipsoid in N dimensional space. PCA corresponds to a coordinate rotation from the natural sensor space axis to a novel axis basis formed by the principal... 

Note The interpretation of the matrix as the covariance matrix of the errors in x has important applications. The value of any estimate is greatly enhanced if its accuracy is known. is also very useful in initial design and development, as it can be calculated before the estimator is implemented. can be used to study measurement placement and what type and accuracy of information is actually needed. A... 

Enew can be calculated as function of B, and the covariance matrix for the old case, Eold, by the following expression ... 

The solution of the minimization problem again simplifies to updating steps of a static Kalman filter. For the linear case, matrices A and C do not depend on x and the covariance matrix of error can be calculated in advance, without having actual measurements. When the problem is nonlinear, these matrices depend on the last available estimate of the state vector, and we have the extended Kalman filter. 

Only a few publications in the literature have dealt with this problem. Almasy and Mah (1984) presented a method for estimating the covariance matrix of measured errors by using the constraint residuals calculated from available process data. Darouach et al. (1989) and Keller et al. (1992) have extended this approach to deal with correlated measurements. Chen et al. (1997) extended the procedure further, developing a robust strategy for covariance estimation, which is insensitive to the presence of outliers in the data set. 

The indirect method uses Eq. (10.9) to estimate F. This procedure requires the value of the covariance matrix, , which can be calculated from the residuals using the balance equations and the measurements. 

Four samples from a Polynesian island gave the lead isotope compositions given in Table 4.3. Calculate the mean and standard deviation vectors, the covariance matrix and the correlation coefficient between the two isotope ratios. 

The relevant mass balance equations have been written in Section 5.1.1 and the matrix A is given in Table 5.1. The standard deviations on intensities are calculated, i.e., for mass 142 as 0.1 x 207 = 1.439 and arranged to form the diagonal matrix S. Then the correlation matrix R is formed out of 1 on the diagonal and 0.85 everywhere else, and, from equation (4.2.18) the covariance matrix Wt is calculated as... 

The basis for calculating the correlation between two variables xj and xk is the covariance covariance matrix (dimension m x m), which is a quadratic, symmetric matrix. The cases j k (main diagonal) are covariances between one and the same variable, which are in fact the variances o-jj of the variables Xj for j = 1,..., m (note that in Chapter 1 variances were denoted as variance—covariance matrix (Figure 2.7). Matrix X refers to a data population of infinite size, and should not be confused with estimations of it as described in Section 2.3.2, for instance the sample covariance matrix C. 

Highly correlating (collinear) variables make the covariance matrix singular, and consequently the inverse cannot be calculated. This has important consequences on the applicability of several methods. Data from chemistry often contain collinear variables, for instance the concentrations of similar elements, or IR absorbances at neighboring wavelengths. Therefore, chemometrics prefers methods that do not need the inverse of the covariance matrix, as for instance PCA, and PLS regression. The covariance matrix becomes singular if... 

Points with a constant Euclidean distance from a reference point (like the center) are located on a hypersphere (in two dimensions on a circle) points with a constant Mahalanobis distance to the center are located on a hyperellipsoid (in two dimensions on an ellipse) that envelops the cluster of object points (Figure 2.11). That means the Mahalanobis distance depends on the direction. Mahalanobis distances are used in classification methods, by measuring the distances of an unknown object to prototypes (centers, centroids) of object classes (Chapter 5). Problematic with the Mahalanobis distance is the need of the inverse of the covariance matrix which cannot be calculated with highly correlating variables. A similar approach without this drawback is the classification method SIMCA based on PC A (Section 5.3.1, Brereton 2006 Eriksson et al. 2006). 

PCA components with small variances may only reflect noise in the data. Such a plot looks like the profile of a mountain after a steep slope a more flat region appears that is built by fallen, deposited stones (called scree). Therefore, this plot is often named scree plot so to say, it is investigated from the top until the debris is reached. However, the decrease of the variances has not always a clear cutoff, and selection of the optimum number of components may be somewhat subjective. Instead of variances, some authors plot the eigenvalues this comes from PCA calculations by computing the eigenvectors of the covariance matrix of X note, these eigenvalues are identical with the score variances. 

Any number of desired internal coordinates St (t - 1, Nt) can be calculated from the final Cartesian position vectors rjj1, most conveniently by known vector formulae. For the transformation of the covariance matrix of the Cartesian coordinates into that of the internal coordinates, the derivatives of any particular internal coordinate are required with respect to all Cartesian coordinates that actually participate in the motion of this internal coordinate ... 

Although the r/E-fit and the p-Kr r, (-rM) fit are not equivalent (the former determines three more variables), it could be shown [55] that the molecular structures determined by the r/e-fit and the r -fit are strictly identical, including the covariance matrix. This is due to the specific form of the Jacobian matrix X of the coupled least-squares problem r/ , which permits a decomposition by a non-singular transformation into a smaller least-squares problem rM plus a subsequent direct calculation of the constant rovib contributions Eg. The r -part of the problem alone determines the molecular structure which must then be used (including the covariance matrix of the structural parameters) for the calculation of the contributions Eg. When rotational constants of new isotopomers are to be predicted from the structure determined, the r/E-method performs much better than the r -method due to the presence of the additional rovib parameters . ... 

---