Discriminant Fisher

The approach of Fisher (1938) was originally proposed for discriminating two populations (binary classification), and later on extended to the case of more than two groups (Rao 1948). Here we will first describe the case of two groups, and then extend to the more general case. Although this method also leads to linear functions for classification, it does not explicitly require multivariate normal distributions of the groups with equal covariance matrices. However, if these assumptions are not... 

In case of two groups, the Fisher method transforms the multivariate data to a univariate discriminant variable such that the transformed groups are separated as much as possible. For this transformation, a linear combination of the original x-variables is used, in other words a latent variable. 

Figure 5.6 visualizes the idea of Fisher discriminant analysis for two groups in the two-dimensional case. The group centers (filled symbols) are projected on the discriminant variable, giving yi and y2 the mean of both is the classification threshold y0. 

The validity discriminant discussed in this section is the descendant of an earlier cluster validity measure used by Gunderson ( ) to assess the quality of cluster configurations obtained in an application of the Fuzzy ISODATA algorithms. It is closely related to a method suggested by Sneath ( ) for testing the distinctness, i.e. separation, of two clusters, and also borrows from the ideas of Fisher s linear discriminant theory (see chapt. 4, Duda and Hart,(2 0). The validity discriminant attempts to measure the separation between the classes of a cluster configuration usually, but not necessarily, obtained by application of the FCV algorithms. A brief description follows ... 

Kennedy JW, Kaiser GC, Fisher LD, et al. Multivariate discriminant analysis of the clinical and angiographic predictors of operative mortality form the collaborative study in coronary artery surgery (CASS). J Thorac Car-diovasc Surg 1980 80 876-887. 

When feature selection is used to simplify, because of the large number of variables, methods must be simple. The univariate criterion of interclass variance/intraclass variance ratio (in the different variants called Fisher weights variance weights or Coomans weights is simple, but can lead to the elimination of variables with some discriminant power, either separately or, more important, in connection with other variables (Fig. 36). 

Fig. 36a-d. Some examples of single variables (a, b) or variable pairs (c, d) whose discriminant power cannot be detected by univariate Fisher weights. The transforms y + x (a) and y/x (b) have high univariate weight... 

Many types of classifiers are based on linear discriminants of the form shown in (1). They differ with regard to how the weights are determined. The oldest form of linear discriminant is Fisher s linear discriminant. To compute the weights for the Fisher linear discriminant, one must estimate the correlation between all pairs of genes that were selected in the feature selection step. The study by Dudoit et al. indicated that Fisher s linear discriminant did not perform well unless the number of selected genes was small relative to the number of samples. The reason is that in other cases there are too many correlations to estimate and the method tends to be unstable and over-fit the data. 

An example of PCA compression is made using the classic Fisher s Irises data set.29 Table 8.2 lists part of a data set containing four descriptors (X-variables) for each of 150 different iris samples. Note that these iris samples fall into three known classes Setosa, Verginica, and Versicolor. From Table 8.2, it is rather difficult to determine whether the four X-variables can be useful for discriminating between the three known classes. 

Discriminant classifiers. The two most important discriminant classifiers for material analysis using spectroscopic imaging systems are the Fisher linear discriminant classifier (FLDC) and the quadratic discriminant classifier (QDC). Other classfiers, such as the classical linear disriminant classifier (LDC), have frequently exhibited an inferior performance. 

Most traditional approaches to classification in science are called discriminant analysis and are often also called forms of hard modelling . The majority of statistically based software packages such as SAS, BMDP and SPSS contain substantial numbers of procedures, referred to by various names such as linear (or Fisher) discriminant analysis and canonical variates analysis. There is a substantial statistical literature in this area. 

In supervised pattern recognition, a major aim is to define the distance of an object from the centre of a class. There are two principle uses of statistical distances. The first is to obtain a measurement analogous to a score, often called the linear discriminant function, first proposed by the statistician R A Fisher. This differs from the distance above in that it is a single number if there are only two classes. It is analogous to the distance along line 2 in Figure 4.26, but defined by... 

Woods JH, Winger GD, France CP (1987) Reinforcing and discriminative stimulus effects of cocaine analysis of pharmacological mechanisms. In Fisher S, Raskin A, Uhlenhuth EH (Eds), Cocaine Clinical and Biobehavioral Aspects, pp. 21-65. Oxford UP, Oxford. 

---