Linear discriminant analysis covariance

When all are considered equal, this means that they can be replaced by S, the pooled variance-covariance matrix, which is the case for linear discriminant analysis. The discrimination boundaries then are linear and is given by... 

This supervised classification method, which is the most used, accepts a normal multivariate distribution for the variables in each population ((Ai,..., A ) Xi) ), and calculates the classification functions minimising the possibility of incorrect classification of the observations of the training group (Bayesian type rule). If multivariate normality is accepted and equality of the k covariance matrices ((Ai,..., Xp) NCfti, X)), Linear Discriminant Analysis (LDA) calculates... 

At a 5% level of significance, the critical value for from tables is 7.8. Our value of 37.3 far exceeds this critical value, and the null hypothesis is rejected. We may assume, therefore, that the two groups of samples are unlikely to have similar parent populations and, hence, similar variance-covariance matrices. It is not surprising, therefore, that the linear discriminant analysis model was inferior to the quadratic scheme in classification. 

Linear discriminant analysis (LDA) is a classification method that uses the distance between the incoming sample and the class centroid to classify the sample. For LDA using Mahalanobis distances, the classification metric uses the pooled variance-covariance matrix to weight the Mahalanobis distance ) between the incoming... 

The particular Bayesian classifier that we consider in this chapter is Bayesian linear discriminant analysis (BLDA). For BLDA one assumes that the class covariance matrices Sr are equal. A pooled covariance matrix is constructed as follows... 

For most applications in the "omics" fields, even the most simple multivariate techniques such as Linear Discriminant Analysis (LDA) cannot be applied directly. From Equation 2 it is clear that an inverse of the the covariance matrix 2 needs to be calculated, which is impossible in cases where the number of variables exceeds the number of samples. In practice, the number of samples is nowhere near the number of variables. For QDA, the situation is even worse to allow a stable matrix inversion, every single class should have at least as many samples as variables (and preferably quite a bit more). A common approach is to compress the information in the data into a low number of latent variables (LVs), either using PCA (leading... 

Parametric discriminant development methods employ the mean vectors and covariance matrices of each class of data to develop the separating discriminant. Linear discriminant analysis (LDA), based on Bayesian statistics, generates a linear discriminant function with the following form ... 

ECVA consists of several steps that finally lead to the actual class membership prediction. First there is a matrix compression step using SVD (if number of variables exceed number of samples n>m)). This is followed by the calculation of covariance matrices, both for the within class variatiOTi and between class variation. Subsequently, a PLS is performed between these matrices and the class relationship and finally a linear discriminant analysis is performed on the PLS results. This indicates that ECVA is a very different classification technique than PLS-DA described above, which is why we have chosen to show how it performs in addition to the above-menti(Mied techniques. 

The right plot in Figure 5.3 shows a linear discrimination of three groups. Here all three groups have the same prior probability, but their covariance matrices are not equal (different shape and orientation). The resulting mle is no longer optimal in the sense defined above. An optimal rule, however, could be obtained by quadratic discriminant analysis which does not require equality of the group covariances. 

Another parametric routine implements a discriminant function by the method commonly called linear discriminant function analysis. It is nearly identical to the linear Bayesian discriminant, except that instead of using the covariance matrix, the sum of cross-products matrix is used. Results obtained with the routine are ordinarily very similar to those obtained using the linear Bayes routine. The routine implemented as LDFA is a highly modified version of program BMD04M taken from the Biomedical Computer Programs Package (47). 

The linear discriminant function is a most commonly used classification technique and it is available with all the most popular statistical software packages. It should be borne in mind, however, that it is only a simplification of the Bayes classifier and assumes that the variates are obtained from a multivariate normal distribution and that the groups have similar covariance matrices. If these conditions do not hold then the linear discriminant function should be used with care and the results obtained subject to careful analysis. 

---