Supervised discriminant analysis

Discriminant emalysis is a supervised learning technique which uses classified dependent data. Here, the dependent data (y values) are not on a continuous scale but are divided into distinct classes. There are often just two classes (e.g. active/inactive soluble/not soluble yes/no), but more than two is also possible (e.g. high/medium/low 1/2/3/4). The simplest situation involves two variables and two classes, and the aim is to find a straight line that best separates the data into its classes (Figure 12.37). With more than two variables, the line becomes a hyperplane in the multidimensional variable space. Discriminant analysis is characterised by a discriminant function, which in the particular case of hnear discriminant analysis (the most popular variant) is written as a linear combination of the independent variables ... 

Most of the supervised pattern recognition procedures permit the carrying out of stepwise selection, i.e. the selection first of the most important feature, then, of the second most important, etc. One way to do this is by prediction using e.g. cross-validation (see next section), i.e. we first select the variable that best classifies objects of known classification but that are not part of the training set, then the variable that most improves the classification already obtained with the first selected variable, etc. The results for the linear discriminant analysis of the EU/HYPER classification of Section 33.2.1 is that with all 5 or 4 variables a selectivity of 91.4% is obtained and for 3 or 2 variables 88.6% [2] as a measure of classification success. Selectivity is used here. It is applied in the sense of Chapter... 

Next, supervised-learning pattern recognition methods were applied to the data set. The 111 bonds from these 28 molecules were classified as either breakable (36) or non-breakable (75), and a stepwise discriminant analysis showed that three variables, out of the six mentioned above, were particularly significant resonance effect, R, bond polarity, Qa, and bond dissociation energy, BDE. With these three variables 97.3% of the non-breakable bonds, and 86.1% of the breakable bonds could be correctly classified. This says that chemical reactivity as given by the ease of heterolysis of a bond is well defined in the space determined by just those three parameters. The same conclusion can be drawn from the results of a K-nearest neighbor analysis with k assuming any value between one and ten, 87 to 92% of the bonds could be correctly classified. 

Current methods for supervised pattern recognition are numerous. Typical linear methods are linear discriminant analysis (LDA) based on distance calculation, soft independent modeling of class analogy (SIMCA), which emphasizes similarities within a class, and PLS discriminant analysis (PLS-DA), which performs regression between spectra and class memberships. More advanced methods are based on nonlinear techniques, such as neural networks. Parametric versus nonparametric computations is a further distinction. In parametric techniques such as LDA, statistical parameters of normal sample distribution are used in the decision rules. Such restrictions do not influence nonparametric methods such as SIMCA, which perform more efficiently on NIR data collections. 

Supervised methods rely on some prior training of the system with objects known to belong to the class they define. Such methods can be of the discriminant or modeling types.11 Discriminant methods split the pattern space into as many regions as the classes encompassed by the training set and establish bounds that are shared by the spaces. These methods always classify an unknown sample as a specific class. The most common discriminant methods include discriminant analysis (DA),12 the K-nearest neighbor... 

Supervised learning methods - multivariate analysis of variance and discriminant analysis (MVDA) - k nearest neighbors (kNN) - linear learning machine (LLM) - BAYES classification - soft independent modeling of class analogy (SIMCA) - UNEQ classification Quantitative demarcation of a priori classes, relationships between class properties and variables... 

Questions of type (2.1) may be answered by analysis of variance or by discriminant analysis. All these methods may be found under the name supervised learning or supervised pattern recognition methods. In the sense of question (2.1.3) one may speak of supervised classification or even better of re-classification methods. In situations of type (2.2) methods from the large family of regression methods are appropriate. 

The goal of classification, also known as discriminant analysis or supervised learning, is to obtain rules that describe the separation between known groups of observations. Moreover, it allows the classification of new observations into one of the groups. We denote the number of groups by Z and assume that we can describe our experiment in each population icj by a / -dimensional random variable Xj with distribution function (density) fj. We write pj for the membership probability, i.e., the probability for an observation to come from icj. 

The main goal of this section is to provide a summary of several of the most widely used multivariate procedures in food authentication out of the vast array currently available. These are included in well-known computer packages such as BMDP, IMSL, MATLAB, NAG, SAS, SPSS and STATISTIC A. The first three subsections describe unsupervised procedures, also called exploratory data analysis, that can reveal hidden patterns in complex data by reducing data to more interpretable information, to emphasize the natural grouping in the data and show which variables most strongly influence these patterns. The fourth and fifth subsections are focused on the supervised procedures of discriminant analysis and regression. The former produces good information when applied under the strictness of certain tests, whereas the latter is mainly used when the objective is calibration. 

Analyses of the above type belong to the category of unsupervised learning , whereas discriminant analysis falls into the supervised analyses of multivariate... 

Table 7.1 Authentication of the geographical origin of virgin olive oil samples comparative results of SEXIA expert system, neural networks and the supervised chemometric procedure of stepwise linear discriminant analysis. Samples collected in the regions of Jaen (Spain)...

Table 7.2 Authentication of mono varietal virgin olive oils comparative results of fuzzy logic algorithms (Calvente and Aparicio, 1995) and the supervised chemometric procedure of linear discriminant analysis. Chemical compounds used linolenic acid, 24-methylen-cycloarthanol sterol and copaene hydrocarbon...

This supervised classification method, which is the most used, accepts a normal multivariate distribution for the variables in each population ((Ai,..., A ) Xi) ), and calculates the classification functions minimising the possibility of incorrect classification of the observations of the training group (Bayesian type rule). If multivariate normality is accepted and equality of the k covariance matrices ((Ai,..., Xp) NCfti, X)), Linear Discriminant Analysis (LDA) calculates... 

Using this approach the first multiplexed simultaneous detection of six different DNA sequences, corresponding to different strains of the Escherichia coli bacterium, each labeled with a different commercially available dye label (ROX, HEX, FAM, TET, Cy3, or TAMRA) was reported [52]. In this study, both exploratory discriminant analysis and supervised learning, by partial least squares (PLS) regression, were used and the ability to discriminate whether a particular labeled... 

---