Linear discriminate analysis

The previously mentioned data set with a total of 115 compounds has already been studied by other statistical methods such as Principal Component Analysis (PCA), Linear Discriminant Analysis, and the Partial Least Squares (PLS) method [39]. Thus, the choice and selection of descriptors has already been accomplished. 

Woodruff and co-workers introduced the expert system PAIRS [67], a program that is able to analyze IR spectra in the same manner as a spectroscopist would. Chalmers and co-workers [68] used an approach for automated interpretation of Fourier Transform Raman spectra of complex polymers. Andreev and Argirov developed the expert system EXPIRS [69] for the interpretation of IR spectra. EXPIRS provides a hierarchical organization of the characteristic groups that are recognized by peak detection in discrete ames. Penchev et al. [70] recently introduced a computer system that performs searches in spectral libraries and systematic analysis of mixture spectra. It is able to classify IR spectra with the aid of linear discriminant analysis, artificial neural networks, and the method of fe-nearest neighbors. 

This approach did not seem to be as satisfactory for those sulfamates having heteroatom substituents (hetero-sulfamates). Spillane suggested that the various electronic effects of the hetero-atoms probably introduce an additional variable that is apparently absent, or constant, for the carbosulfamates. Because molecular connectivity correlates structure with molecular volume and electronic effects, Spillane included molecular connectivity, (computed for the entire molecule, RNHSOO to the four variables, x, y, z, and V, and applied the statistical technique of linear-discrimination analysis to 33 heterosulfamates (10 sweet, 23 not sweet). A correlation of >80% was obtained for the x, z, x subset 5 of the 33... 

A first distinction which is often made is that between methods focusing on discrimination and those that are directed towards modelling classes. Most methods explicitly or implicitly try to find a boundary between classes. Some methods such as linear discriminant analysis (LDA, Sections 33.2.2 and 33.2.3) are designed to find explicit boundaries between classes while the k-nearest neighbours (A -NN, Section 33.2.4) method does this implicitly. Methods such as SIMCA (Section 33.2.7) put the emphasis more on similarity within a class than on discrimination between classes. Such methods are sometimes called disjoint class modelling methods. While the discrimination oriented methods build models based on all the classes concerned in the discrimination, the disjoint class modelling methods model each class separately. 

Fig. 33.1. Canonical variate plot for three classes with different thyroid status. The boundaries are obtained by linear discriminant analysis [2].

We also make a distinction between parametric and non-parametric techniques. In the parametric techniques such as linear discriminant analysis, UNEQ and SIMCA, statistical parameters of the distribution of the objects are used in the derivation of the decision function (almost always a multivariate normal distribution... 

In the method of linear discriminant analysis, one therefore seeks a linear function of the variables, D, which maximizes the ratio between both variances. Geometrically, this means that we look for a line through the cloud of points, such that the projections of the points of the two groups are separated as much as possible. The approach is comparable to principal components, where one seeks a line that explains best the variation in the data (see Chapter 17). The principal component line and the discriminant function often more or less coincide (as is the case in Fig. 33.8a) but this is not necessarily so, as shown in Fig. 33.8b. 

So fiir, we have described only situations with two classes. The method can also be applied to K classes. It is then sometimes called descriptive linear discriminant analysis. In this case the weight vectors can be shown to be the eigenvectors of the matrix ... 

When all are considered equal, this means that they can be replaced by S, the pooled variance-covariance matrix, which is the case for linear discriminant analysis. The discrimination boundaries then are linear and is given by... 

Most of the supervised pattern recognition procedures permit the carrying out of stepwise selection, i.e. the selection first of the most important feature, then, of the second most important, etc. One way to do this is by prediction using e.g. cross-validation (see next section), i.e. we first select the variable that best classifies objects of known classification but that are not part of the training set, then the variable that most improves the classification already obtained with the first selected variable, etc. The results for the linear discriminant analysis of the EU/HYPER classification of Section 33.2.1 is that with all 5 or 4 variables a selectivity of 91.4% is obtained and for 3 or 2 variables 88.6% [2] as a measure of classification success. Selectivity is used here. It is applied in the sense of Chapter... 

D. Coomans, M. Jonckheer, D.L. Massait, I. Broeckaert and P. Blockx, The application of linear discriminant analysis in the diagnosis of thyroid diseases. Anal. Chim. Acta, 103 (1978) 409-415. 

W. Wu, Y. Mallet, B. Walczak, W. Penninckx, D.L. Massart, S. Heuerding and F. Erni, Comparison of regularized discriminant analysis, linear discriminant analysis and quadratic discriminant analysis, applied to NIR data. Anal. Chim. Acta, 329 (1996) 257-265. 

While principal components models are used mostly in an unsupervised or exploratory mode, models based on canonical variates are often applied in a supervisory way for the prediction of biological activities from chemical, physicochemical or other biological parameters. In this section we discuss briefly the methods of linear discriminant analysis (LDA) and canonical correlation analysis (CCA). Although there has been an early awareness of these methods in QSAR [7,50], they have not been widely accepted. More recently they have been superseded by the successful introduction of partial least squares analysis (PLS) in QSAR. Nevertheless, the early pattern recognition techniques have prepared the minds for the introduction of modem chemometric approaches. 

A drawback of the method is that highly correlating canonical variables may contribute little to the variance in the data. A similar remark has been made with respect to linear discriminant analysis. Furthermore, CCA does not possess a direction of prediction as it is symmetrical with respect to X and Y. For these reasons it is now replaced by two-block or multi-block partial least squares analysis (PLS), which bears some similarity with CCA without having its shortcomings. 

The application of linear discriminant analysis of the 16 amino acids foimd in Spanish honeys established both botanical and geographical differences (Perez Arquillue and Herrera Marteache, 1987). Gas chromatography (GC) analyses of free amino acids showed obvious differences when honey samples from the UK, Argentina, Australia, and Canada were compared (Gilbert et al., 1981). Pawlowska and Armstrong (1994)... 

The NMR spectra using PCA and Linear Discriminant Analysis (LDA) obtained for instant spray dried coffees from a number of different manufacturers demonstrated [8] that the concentration of the extracted molecules is generally high enough for clear detection. The compound 5-(hydroxymethy)-2-furaldehyde was identified as the primary marker of differentiation between two groups of coffees. This method may be used to determine whether a fraudulent retailer is selling an inferior quality product marked as being from a reputable manufacturer [8]. 

Several additional instrumental techniques have also been developed for bacterial characterization. Capillary electrophoresis of bacteria, which requires little sample preparation,42 is possible because most bacteria act as colloidal particles in suspension and can be separated by their electrical charge. Capillary electrophoresis provides information that may be useful for identification. Flow cytometry also can be used to identify and separate individual cells in a mixture.11,42 Infrared spectroscopy has been used to characterize bacteria caught on transparent filters.113 Fourier-transform infrared (FTIR) spectroscopy, with linear discriminant analysis and artificial neural networks, has been adapted for identifying foodbome bacteria25,113 and pathogenic bacteria in the blood.5... 

There are many classification methods apart from linear discriminant analysis (Derde et al. [1987] Frank and Friedman [1989] Huberty [1994]). Particularly worth mentioning are the SIMCA method (Soft independent modelling of class analogies) (Wold [1976] Frank [1989]), ALLOC (Coomans et al. [1981]), UNEQ (Derde and Massart [1986]), PRIMA (Juricskay and Veress [1985] Derde and Massart [1988]), DASCO (Frank [1988]), etc. 

Coomans D, Massart DL, Broeckaert I (1981) Potential methods in pattern recognition. A combination of ALLOC and statistical linear discriminant analysis. Anal Chim Acta 133 215... 

---