Linear classifiers

Prior to the actual classification, the FLDC performs a linear mapping to a lower dimensional subspace optimised for class separability, based on the between-class scatter and the within-class scatter of the training set. In classification, each sample is assigned to the class giving the highest log-likelihood using a linear classifier. 

The training of the perceptron as a linear classifier then follows the following... 

Linear discriminant analysis (LDA) is used in statistics and machine learning methods to find the best linear combination of descriptors that distinguish two or more classes of objects or events, and, in the present case, to distinguish between substrates and nonsubstrates of P-gp. A linear classifier achieves this by making a classification decision based on the value of the linear combination of descriptors. 

Figure 5.12 Simple two-group, bivariate data set that is not linearly separable by a single function. Lines shown are the linear classifiers from the two units in the first layer of the multi-layer system shown in Figure 5.13...

The above formulation is known as linear NPPC. When the patterns are not linearly separable then one can use nonlinear NPPC. The linear NPPC can be extended to nonlinear classifiers by applying the kernel trick [18]. For nonlinearly separable patterns, the input data is first mapped into a higher dimensional feature space by some kernel function. In the feature space it implements a linear classifier which correspond a nonlinear separating surface in the input space. To apply this transformation, let k(.,.) be any nonlinear kernel function and define the augmented matrix ... 

FIGURE 16. Scheme of the learning machine Training of an adaptive, binary, linear classifier. 

Classification of a d-diraensional pattern vector x by a Linear classifier is performed by computing the scalar product s (Chapter 1.3, equation (2)),... 

For this calculation a set of n pattern vectors with known class memberships is necessary. For a linear classifier the decision function s has the general form... 

Application of a.multicategory piecewise-linear classifier to the interpretation of mass spectra was of "varying success" C88, 89, 1173-Recognition of a C=C double bond in a linearly inseparable data set of mass spectra required 3 to 5 weight vectors to obtain 78 to 87 % predictive ability (75 % was obtained with a single weight vector for the same data set). The lack of a satisfactory theory of piecewise-linear classifiers and rather high computational expenses have prevented up to now broader applications of this classification method. 

The KNN-method is the method of choice if the cluster structure is complex and a linear classifier fails. Because of the large computational requirements necessary for KNN-classifications the method is not suitable for a large number of unknowns or a large data set of known patterns. 

The perceptron consists of two main parts. In the first part a new pattern vector y is calculated from each original pattern vector x. Each component y is a linear combination of several randomly selected components from the original pattern. The second part is an application of a binary, linear classifier and all methods from Chapter 2 may be used. 

Once the parameters of the Gaussian probability density functions for all classes are known, the density at any location can be calculated and an unknown pattern can be classified by the Bayes rule or by the maximum likelihood method. A binary classification with equal covariance matrices for both classes can be reduced in this way to a linear classifier C87, 317, 396D. 

Two problems arise when equation (106) is used to evaluate the significance of a linear classifier. 

The perceptron and the learning machine are iterative algorithms for finding linear classifiers. These methods played an important role in the pioneering time of chemometrics but are not often applied today because they have a number of drawbacks,... 

SVM can also be used to separate classes that cannot be separated with a linear classifier (Figure 2, left). In such cases, the coordinates of the objects are mapped into a feature space using nonlinear functions called feature functions ( ). The feature space is a high-dimensional space in which the two classes can be separated with a linear classifier (Figure 2, right). 

Figure 14 Using the linear classifier defined by the hyperplane H, the pattern is predicted to belong to the class —1.

A hyperplane w x + b = 0 can be denoted as a pair (w, b). A training set of patterns is linearly separable if at least one linear classifier exists defined by the pair (w, b), which correctly classifies all training patterns (see Figure 15). All patterns from class +1 are located in the space region defined by w X + > 0, and all patterns from class —1 are located in the space region... 

Consider a group of linear classifiers (hyperplanes) defined by a set of pairs (w, b) that satisfy the following inequalities for any pattern x, in the training set ... 

In general, for each linearly separable training set, one can find an infinite number of hyperplanes that discriminate the two classes of patterns. Although all these linear classifiers can perfectly separate the learning patterns, they are not all identical. Indeed, their prediction capabilities are different. A hyperplane situated in the proximity of the border +1 patterns will predict as —1 all new -hi patterns that are situated close to the separation hyperplane but in the —1 region (w x- -fi<0). Conversely, a hyperplane situated in the proximity of the border —1 patterns will predict as -hi all new —1 patterns situated close to the separation hyperplane but in the - -1 region (w x -F > 0). It is clear that such classifiers have little prediction success, which led to the idea... 

Figure 19 In a plane, all combinations of three points from two classes can be separated with a line. Four points cannot be separated with a linear classifier.

---