Covariance Cross correlations

It can be shown that all symmetric matrices of the form X X and XX are positive semi-definite [2]. These cross-product matrices include the widely used dispersion matrices which can take the form of a variance-covariance or correlation matrix, among others (see Section 29.7). 

A theorem, which we do not prove here, states that the nonzero eigenvalues of the product AB are identical to those of BA, where A is an nxp and where B is a pxn matrix [3]. This applies in particular to the eigenvalues of matrices of cross-products XX and X which are of special interest in data analysis as they are related to dispersion matrices such as variance-covariance and correlation matrices. If X is an nxp matrix of rank r, then the product X X has r positive eigenvalues in A and possesses r eigenvectors in V since we have shown above that ... 

The adjustment of measurements to compensate for random errors involves the resolution of a constrained minimization problem, usually one of constrained least squares. Balance equations are included in the constraints these may be linear but are generally nonlinear. The objective function is usually quadratic with respect to the adjustment of measurements, and it has the covariance matrix of measurements errors as weights. Thus, this matrix is essential in the obtaining of reliable process knowledge. Some efforts have been made to estimate it from measurements (Almasy and Mah, 1984 Darouach et al., 1989 Keller et al., 1992 Chen et al., 1997). The difficulty in the estimation of this matrix is associated with the analysis of the serial and cross correlation of the data. 

PCA is a statistical technique that has been used ubiquitously in multivariate data analysis." Given a set of input vectors described by partially cross-correlated variables, the PCA will transform them into a set that is described by a smaller number of orthogonal variables, the principle components, without a significant loss in the variance of the data. The principle components correspond to the eigenvectors of the covariance matrix, m, a symmetric matrix that contains the variances of the variables in its diagonal elements and the covariances in its off-diagonal elements (15) ... 

A much weaker property, implied by statistical independence but not implying it, is vanishing of the covariance, the second cross central moment, sometimes called the cross correlation,... 

McCabe techniques of variable reduction are based on the calculation of the conditional covariance (or correlation) matrix of the excluded variables McCabe, 1984 1410 /id. This matrix represents the residual information left in the variable that are not selected, after the effect of the most relevant variables has been removed. It is a square symmetric matrix of order q = p — k, vhere p is the total number of variables and k is the number of retained variables, and is derived from the covariance (correlation) matrix of the retained variables Sj (of size k x k), the covariance (correlation) matrix of the deleted variables Sjj (of size q X q), and the cross-covariance (correlation) matrix bet veen the t vo sets of variables Srd (of size k X q) ... 

While the estimates of the autocorrelation coefficients for the Cg time series (lower rows in 1 to ordy change slightly, the estimates the autocorrelation coefficients for the Benzene time series (upper rows in to 3) are clearly affected since three parameters are dropped from the model. The remaining coefficients are affected, too. In particular, the lagged cross-correlations to the Cg time series change from 1.67 to 2.51 and from -2.91 to -2.67 (right upper entries in 1 and This confirms the serious effect of even unobtrusive outliers in multivariate times series analysis. By incorporating the outliers effects, the model s AIC decreases from -4.22 to -4.72. Similarly, SIC decreases from -4.05 to -4.17. The analyses of residuals. show a similar pattern as for the initial model and reveal no serious hints for cross- or auto-correlation. i Now, the multivariate Jarque-Bera test does not reject the hypothesis of multivariate normally distributed variables (at a 5% level). The residuals empirical covariance matrix is finally estimated as... 

The cross-correlation between two measured biosignals x(n) and y(n) is defined statistically as Eyx(IO = Ely(n)x(n + h)], where the operator E represents the statistical expectation or mean and k is the amount of time signal jc(n) is delayed with respect to y(n). Given two time sequences of x( ) and y(n) each of N points, the commonly used estimate of cross-covariance c,Jik) is as follows ... 

In contrast to the principal component (PCA)-based approach for detection and discrimination, the cross-correlation or matched-filter approach is based on a priori knowledge of the shape of the deterministic signal to be detected. However, the PCA-based method also requires some initial data (although the data could be noisy, and the detector does not need to know a priori the label or the class of the diHerent signals) to evaluate the sample covariance matrix K and its eigenvectors. In this sense, the PCA-based detector operates in an unsupervised mode. Further, the matched-filter approach is optimal only when the interfering noise is white Gaussian. When the noise is colored, the PCA-based approach will he preferred. 

The cross-covariance frmction for the two time series does not depend on t. Joint stationarity implies that the cross-correlation function can be written as... 

Estimating the Autocovariance and Cross-Covariance and Correlation Functions... 

The cross-correlation plot shown in Eig. 5.4 between the mean summer and spring temperatures has a similar format to the previously considered autocorrelation plot. The confidence interval is, as was previously noted, the same as for the autocorrelation plot. The salient feature is the 4 largest lags at —20, —16, 3, and 4. At this point, it would be useful to comment briefly about the meaning of these values. Since the formula for computing the cross-covariance can be written as or equivalently yt = Xf we can see that positive values correspond to a relationship between past values of x (or in our case, the mean spring temperature)... 

Autocorrelation Coefficient n The auto covariance normalized by the product of the standard deviations of the two sections from the single random variable sequence used to calculate the autocovariance. In other words the autocorrelation coefficient is the cross-correlation coefficient of two sub-sequences of the same random variable. It is probably the most commonly used measure of the correlation between two sections of a single random variable sequence. It is often simply but incorrectly referred to as the autocorrelation, which is the un-normalized expectation value of the product of the two sequence sections. The autocorrelation coefficient of the two subsequences of random variable, X, is often denoted by Pxx(ii>T) where i is the starting index of the second section, and T is the length of the sections. The precise mathematical definition of autocorrelation coefficient of two random variable sequence sections is given by ... 

Cross-Correlation Coefficient n The cross-covariance normalized hy the product of the standard deviations of the two sections from the two random variable sequences used to calculate the cross-covariance. The cross-correlation coefficient is a widely used measure of the correlation between two sections of two different... 

The cross-covariance of a random variable with itself is referred to as the autocovariance, which is often used interchangeably and confused with the autocorrelation which is actually the cross-correlation of a random variable with itself. The cross-covariance is related to and often confused with the cross-correlation. 

Figure 2 (A) A schematic drawing of an array of one-dimensional spectra of a homo-nuclear three-spin system with incremented evolution time q. We suppose that the magnetizations of the spins labeled with i, j, and k change with time as depicted in (B). Since the way that the magnetizations of / and k change with is somewhat correlated, a covariance cross-peak appears between / and k, as drawn in (Q, whereas the time dependence of the j magnetization is quite different from others, resulting in no appreciable covariance cross-peaks.

In the following section on preprocessing of the data we will show that column-centering of X leads to an interpretation of the sums of squares and cross-products in in terms of the variances-covariances of the columns of X. Furthermore, cos djj> then becomes the coefficient of correlation between these columns. 

Equation 41-A3 can be checked by expanding the last term, collecting terms and verifying that all the terms of equation 41-A2 are regenerated. The third term in equation 41-A3 is a quantity called the covariance between A and B. The covariance is a quantity related to the correlation coefficient. Since the differences from the mean are randomly positive and negative, the product of the two differences from their respective means is also randomly positive and negative, and tend to cancel when summed. Therefore, for independent random variables the covariance is zero, since the correlation coefficient is zero for uncorrelated variables. In fact, the mathematical definition of uncorrelated is that this sum-of-cross-products term is zero. Therefore, since A and B are random, uncorrelated variables ... 

Let us discuss some of the terms in equation 70-20. The simplest way to think about the covariance is to compare the third term of equation 70-20 with the numerator of the expression for the correlation coefficient. In fact, if we divide the last term on the RHS of equation 70-20 by the standard deviations (the square root of the variances) of X and Y in order to scale the cross-product by the magnitudes of the X and Y variables and make the result dimensionless, we obtain... 

LCA and CCK, on the other hand, appear to be strikingly dissimilar. All CCK procedures require at least one quasi-continuous indicator, and if there are none, the investigator has to create such an indicator (e.g., SSMAXCOV procedure). In contrast, LCA does not require continuous indicators and only deals with categorical data. In the case of categorical data, the patterns of interest are usually apparent, so there is no need to summarize the data with correlations. Therefore, LCA evaluates cross-tabulations and compares the number of cases across cells. This shift in representation of the data necessitates other basic changes. For example, LCA operates with proportions instead of covariances and yields tables rather than plots. These differences aside, the two approaches share a lot in common. LCA, like CCK, starts with a set of correlated indicators. It also makes the assumption of zero nuisance covariance-—in the LCA literature this is called the assumption of local independence, and it means that the indicators are presumed to be independent (i.e., uncorrelated) within latent classes. Moreover, LCA and CCK (MAXCOV in particular) use similar procedures for group assignment and both of them involve Bayes s theorem. 

Figure 3 PCA triplot showing the correlation between chemical contaminants in flounder liver (and salinity) and the response of the biomarkers. 12% of the total variance was captured by the covariable age. The horizontal first axis displays 74% of the remaining variation in contaminants in the fish, the vertical second axis another 9%. Only biomarkers that explained 10% or more of the total variance are shown. Biomarkers arrows contaminants in flounder crosses sampling locations filled circles. Abbreviations of substances are explained in Annex 1 and fig 2.

Finally, we allow for cross sectional correlation of the disturbances. Our initial estimate of b is the pooled least squares estimator, 2/3. The estimates of the two variances are. 84444 and. 32222 as before while the cross sectional covariance estimate is... 

---