Quadratic discriminant analysis and related methods

There is still another approach to explain LDA, namely by considering the Mahalanobis distance (see Chapter 30) to a class. All these approaches lead to the same result. The Mahalanobis distance is the distance to the centre of a class taking correlation into account and is the same for all points on the same probability ellipse. For equally probable classes, i.e. classes with the same number of training objects, a smaller Mahalanobis distance to class K than to class L, means that the probability that the object belongs to class K is larger than that it belongs to L. 

The Mahalanobis distance representation will help us to have a more general look at discriminant analysis. The multivariate normal distribution for w variables and class K can be described by 

When all are considered equal, this means that they can be replaced by S, the pooled variance-covariance matrix, which is the case for linear discriminant analysis. The discrimination boundaries then are linear and is given by 

Friedman [12] introduced a Bayesian approach the Bayes equation is given in Chapter 16. In the present context, a Bayesian approach can be described as finding a classification rule that minimizes the risk of misclassification, given the prior probabilities of belonging to a given class. These prior probabilities are estimated from the fraction of each class in the pooled sample  

Equation (33.10) is applied in what is called quadratic discriminant analysis (QDA). The equations can be shown to describe a quadratic boundary separating the regions where is minimal for the classes considered. 

---