Supervised pattern recognition discriminant analysis

Most of the supervised pattern recognition procedures permit the carrying out of stepwise selection, i.e. the selection first of the most important feature, then, of the second most important, etc. One way to do this is by prediction using e.g. cross-validation (see next section), i.e. we first select the variable that best classifies objects of known classification but that are not part of the training set, then the variable that most improves the classification already obtained with the first selected variable, etc. The results for the linear discriminant analysis of the EU/HYPER classification of Section 33.2.1 is that with all 5 or 4 variables a selectivity of 91.4% is obtained and for 3 or 2 variables 88.6% [2] as a measure of classification success. Selectivity is used here. It is applied in the sense of Chapter... 

Current methods for supervised pattern recognition are numerous. Typical linear methods are linear discriminant analysis (LDA) based on distance calculation, soft independent modeling of class analogy (SIMCA), which emphasizes similarities within a class, and PLS discriminant analysis (PLS-DA), which performs regression between spectra and class memberships. More advanced methods are based on nonlinear techniques, such as neural networks. Parametric versus nonparametric computations is a further distinction. In parametric techniques such as LDA, statistical parameters of normal sample distribution are used in the decision rules. Such restrictions do not influence nonparametric methods such as SIMCA, which perform more efficiently on NIR data collections. 

Questions of type (2.1) may be answered by analysis of variance or by discriminant analysis. All these methods may be found under the name supervised learning or supervised pattern recognition methods. In the sense of question (2.1.3) one may speak of supervised classification or even better of re-classification methods. In situations of type (2.2) methods from the large family of regression methods are appropriate. 

If the membership of objects to particular clusters is known in advance, the methods of supervised pattern recognition can be used. In this section, the following methods are explained linear learning machine (LLM), discriminant analysis, A -NN, the soft independent modeling of class analogies (SIMCA) method, and Support Vector Machines (SVMs). 

We haveemployed a variety of unsupervised and supervised pattern recognition methods such as principal component analysis, cluster analysis, k-nearest neighbour method, linear discriminant analysis, and logistic regression analysis, to study such reactivity spaces. We have published a more detailed description of these investigations. As a result of this, functions could be developed that use the values of the chemical effects calculated by the methods mentioned in this paper. These functions allow the calculation of the reactivity of each individual bond of a molecule. 

The most frequently used supervised pattern recognition method is the linear discriminant analysis (LDA), not to be confused with its twin brother canonical correlation analysis (CCA) or canonical variate analysis (CVA). Recently, classification and regression trees (CART) produced surprisingly good results. Artificial neural networks (ANNs) can be applied for both prediction and pattern recognition (supervised and unsupervised). 

Discriminant analysis belongs to supervised pattern recognition and has the aim to assign objects to one of several pre-determined classes. Only the two-class problem will be treated here briefly. Linear discriminant analysis (LDA) uses a latent variable (discriminant variable) in the feature space that maximally separates two classes of objects. A widely used criterion for class separation is the r-value as known from the statistical r-test (Student test) as defined in equations (19)-(21), with niA, ms being the means, va, vb the variances, a, the number of objects of the classes A and B, respectively v is called the pooled variance. 

Linear discriminant analysis (LDA), originally proposed by Fisher in 1936 [8], is the oldest and most studied supervised pattern recognition method. As the name suggests, it is a linear technique, that is the decision boundaries separating the classes in the multidimensional space of the variables are linear surfaces (hyperplanes). From a probabilistic standpoint, it is a parametric method, as its underlying hypothesis is that, for each category, the data follow a multivariate normal distribution. This means that the likelihood in Equation (2), for each class, is defined as... 

Next, supervised-learning pattern recognition methods were applied to the data set. The 111 bonds from these 28 molecules were classified as either breakable (36) or non-breakable (75), and a stepwise discriminant analysis showed that three variables, out of the six mentioned above, were particularly significant resonance effect, R, bond polarity, Qa, and bond dissociation energy, BDE. With these three variables 97.3% of the non-breakable bonds, and 86.1% of the breakable bonds could be correctly classified. This says that chemical reactivity as given by the ease of heterolysis of a bond is well defined in the space determined by just those three parameters. The same conclusion can be drawn from the results of a K-nearest neighbor analysis with k assuming any value between one and ten, 87 to 92% of the bonds could be correctly classified. 

If it is not necessary to know the specific concenirations of the species of interest, bnt to simply know whether such species are present, or not, in a complex sample then, a multivariate pattern recognition method, such as linear discriminant analysis (LDA) or artificial nenral networks (ANNs), may be used to identify the spectral characteristics of the species of interest [10]. Such methods are capable of comparing a large number of variables within a data set, such as intensity, frequency and bandwidth. LDA and ANNs are known as supervised methods because a priori... 

Statistical pattern recognition is based on the statistical nature of signals and extracted features are represented as probability density functions (Schalkoff, 1992). It therefore requires knowledge of a priori probabilities of measurements and features. Statistical approaches include linear discriminant functions, Bayesian functions and cluster analysis and may be unsupervised or supervised. Supervised classifiers require a set of exemplars for each class to be recognized they are used to train the system. Unsupervised learning, on the other hand, does not require an exemplar set. 

---