Covariance robust estimation

Chen, J., Bandoni, A., and Romagnoli, J. A. (1997). Robust estimation of measurement error variance/ covariance from process sampling data. Comput. Chem. Eng. 21, 593-600. 

Pr from the robust estimator still gives the correct answer, as expected. However, the conventional approach fails to provide a good estimate of the covariance even for the case when only one outlier is present in the sampling data. 

The robust estimator still provides a correct estimation of the covariance matrix on the other hand, the estimate J>C> provided by the conventional approach, is incorrect and the signs of the correlated coefficients have been changed by the outliers. 

There exist other estimators for robust covariance or correlation, like S-estimators (Maronna et al. 2006). In general, there are restrictions for robust estimations of the... 

For identifying outliers, it is crucial how center and covariance are estimated from the data. Since the classical estimators arithmetic mean vector x and sample covariance matrix C are very sensitive to outliers, they are not useful for the purpose of outlier detection by taking Equation 2.19 for the Mahalanobis distances. Instead, robust estimators have to be taken for the Mahalanobis distance, like the center and... 

FIGURE 2.13 Concentrations of MgO and Cl in glass vessels samples (Janssen et al. 1998). The plots show the Mahalanobis distances versus the object number the distances are computed using classical (left) and robust (right) estimates for location and covariance. The horizontal lines correspond to the cutoff value Jx fi 975 = 2.72. Using the robust estimates, several outliers are identified. 

There are essentially two different procedures for robust PCA, a method based on robust estimation of the covariance, and a method based on projection pursuit. For the covariance-based procedure the population covariance matrix X has to be... 

Note that since SVD is based on eigenvector decompositions of cross-product matrices, this algorithm gives equivalent results as the Jacobi rotation when the sample covariance matrix C is used. This means that SVD will not allow a robust PCA solution however, for Jacobi rotation a robust estimation of the covariance matrix can be used. 

Methods of robust PCA are less sensitive to outliers and visualize the main data structure one approach for robust PCA uses a robust estimation of the covariance matrix, another approach searches for a direction which has the maximum of a robust variance measure (projection pursuit). 

Unfortunately, the use of these affine equivariant covariance estimators is limited to small to moderate dimensions. To see why, let us again consider the MCD estimator. As explained in Section 6.3.2, if p denotes the number of variables in our data set, the MCD estimator can only be computed if pcovariance matrix of any //-subset has zero determinant. Because h < n, the number of variables p may never be larger than n. A second problem is the computation of these robust estimators in high dimensions. Today s fastest algorithms can handle up to about 100 dimensions, whereas there are fields like chemometrics that need to analyze data with dimensions in the thousands. Moreover the accuracy of the algorithms decreases with the dimension p, so it is recommended that small data sets not use the MCD in more than 10 dimensions. 

Croux, C. and Haesbroeck, G., Principal components analysis based on robust estimators of the covariance or correlation matrix influence functions and efficiencies, Biometrika, 87, 603-618, 2000. 

The most important robust estimators for data reconciliation belong to the class of M-estimators, which are generalizations of the maximum likelihood estimator. Assuming uncorrelated measurement data their covariance matrix becomes diagonal and the generalized DR problem has the form,... 

There are essentially two approaches for robust PCA the first is based on PCA on a robust covariance matrix, which is rather straightforward as the PCs are the eigenvectors of the covariance matrix. Different robust estimators of covariance matrix may be adopted (MVT [92], MVE and MCD [93]) but the decomposition algorithm is the same. The second approach is based on projection pursuit (PP), by using a projection aimed at maximizing a robust measure of scale, that is, in a PP algorithm, the direction with maximum robust variance of the projected data is pursued here different search algorithms are proposed. 

Robust Estimates of Covariance and Multivariate Location and Scatter... 

Several robust estimators of sample covariance like multivariate trimming (MVT) [8], minimum volume ellipsoid (MVE), minimum covariance determinant (MCD) [2], S-estimators [2] and MM-estimators [9] exist. A comprehensive overview of different estimators of location and scatter can be found in [10]. Application of the MCD estimator can be found frequently in the... 

The centring operation required to calculate sample covariance must be robust and thus the LI-median is considered to be a robust estimator of data location. As illustrated by Hubert and Vanden Branden [41], the robust PLS... 

LDA is based on classic estimators of location and covariance, and that is why the method is sensitive to outlying samples decreasing the LDA performance with an increase in the number of incorrectly assigned samples. A relatively simple solution to overcome the LDA lack of robustness is to use robust estimators of data location and covariance instead of their classic counterparts. This can be done using different types of robust estimators and concepts to derive a robust pooled covariance. Different possible schemes to make LDA robust are presented in [43 5]. 

Most techniques for process data reconciliation start with the assumption that the measurement errors are random variables obeying a known statistical distribution, and that the covariance matrix of measurement errors is given. In Chapter 10 direct and indirect approaches for estimating the variances of measurement errors are discussed, as well as a robust strategy for dealing with the presence of outliers in the data set. 

Only a few publications in the literature have dealt with this problem. Almasy and Mah (1984) presented a method for estimating the covariance matrix of measured errors by using the constraint residuals calculated from available process data. Darouach et al. (1989) and Keller et al. (1992) have extended this approach to deal with correlated measurements. Chen et al. (1997) extended the procedure further, developing a robust strategy for covariance estimation, which is insensitive to the presence of outliers in the data set. 

Without outliers. The estimation of P both from the conventional indirect approach PC and from the robust approach 4/r gave similar results when compared with the target covariance matrix (P ... 

Without outliers. As shown by comparison with the target covariance, both the conventional indirect approach and the robust approach give a very good estimation of the covariance in this case ... 

The covariance matrix of measurement errors is a very useful statistical property. Indirect methods can deal with unsteady sampling data, but unfortunately they are very sensitive to outliers and the presence of one or two outliers can cause misleading results. This drawback can be eliminated by using robust approaches via M-estimators. The performance of the robust covariance estimator is better than that of the indirect methods when outliers are present in the data set. 

---