Canonical variates and linear discriminant analysis

In practice, we would prefer an algebraic way to define the boundary. For this purpose, we define line d, perpendicular to a. One can project any object or point on that line. In Fig. 33.6c this is done for point A. The location of A on d is given by its score on d. This score is given by  

When working with standardized data Wg = 0. The coefficients w and W2 are derived in a way described later, such that D = 0 in point O and D 0 for objects belonging to L and 0 for objects of K. This then is the classification rule. 

These weights depend on several characteristics of the data. To understand which ones, let us first consider the univariate case (Fig. 33.7). Two classes, K and L, have to be distinguished using a single variable, Jt,. It is clear that the discrimination will be better when the distance between and (i.e. the mean values, or centroids, of 3 , for classes K and L) is large and the width of the distributions is small or, in other words, when the ratio of the squared difference between means to the variance of the distributions is large. Analytical chemists would be tempted to say that the resolution should be as large as possible. 

When we consider the multivariate situation, it is again evident that the discriminating power of the combined variables will be good when the centroids of the two sets of objects are sufficiently distant from each other and when the clusters are tight or dense. In mathematical terms this means that the between-class variance is large compared with the within-class variances. 

In the method of linear discriminant analysis, one therefore seeks a linear function of the variables, D, which maximizes the ratio between both variances. Geometrically, this means that we look for a line through the cloud of points, such that the projections of the points of the two groups are separated as much as possible. The approach is comparable to principal components, where one seeks a line that explains best the variation in the data (see Chapter 17). The principal component line and the discriminant function often more or less coincide (as is the case in Fig. 33.8a) but this is not necessarily so, as shown in Fig. 33.8b. 

---