Linear separable classes

FIGURE 13.15. Example of two linearly separable classes that can be separated with (a) several hyperplanes, but for which SVM defines (b) a unique separating hyperplane. The margin (M) is the distance between the dashed lines through the support vectors. 

Formally, it can be categorized as a one-layer net with two inputs, and X2, and one output, y, in the sense of a classifier for linearly separable classes. For this classifier, it is valid at time or cycle t that... 

Figure 6.9 An example of non-linearly separable classes of samples in a given subspace (a), and its separation on a higher subspace (b).

Figure 6.10 An example of linearly separable classes where different frontiers (hyperplanes or borders) are possible (a) concept of optimum separated hyperplane, margin and support vectors (b) concept of slack variables (c).

Because we have considered the case of linearly separable classes, each such hyperplane (w, b) is a classifier that correctly separates all patterns from the training set ... 

Furthermore, the pattern structures in a representation space formed from raw input data are not necessarily linearly separable. A central issue, then, is feature extraction to transform the representation of observable features into some new representation in which the pattern classes are linearly separable. Since many practical problems are not linearly separable (Minsky and Papert, 1969), use of linear discriminant methods is especially dependent on feature extraction. 

Like the PLS-DA method, the LDA method is susceptible to overfitting through the use of too many LDs. Furthermore, as in PLS-DA, it assumes that the classes can be linearly separated in the classification space, and its performance suffers if this is not the case. 

Although the development of a SIMCA model can be rather cumbersome, because it involves the development and optimization of J PCA models, the SIMCA method has several distinct advantages over other classification methods. First, it can be more robust in cases where the different classes involve discretely different analytical responses, or where the class responses are not linearly separable. Second, the treatment of each class separately allows SIMCA to better handle cases where the within-class variance structure is... 

Class 1 /, h,g are linearly separable in x andy. Class 2 Variable factor programming... 

The KNN method has several advantages aside from its relative simplicity. It can be used in cases where few calibration data are available, and can even be used if only a single calibration sample is available for some classes. In addition, it does not assume that the classes are separated by linear partitions in the space. As a result, it can be rather effective at handling highly non-linear separation structures. 

The perceptron network is the simplest of these three methods, in that its execution typically involves the simple multiplication of class-specific weight vectors to the analytical profile, followed by a hard limit function that assigns either 1 or 0 to the output (to indicate membership, or no membership, to a specific class). Such networks are best suited for applications where the classes are linearly separable in the classification space. 

Since scope economies are especially hard to quantify, a separate class of optimization models solely dealing with plant loading decisions can be found. For example, Mazzola and Schantz (1997) propose a non-linear mixed integer program that combines a fixed cost charge for each plant-product allocation, a fixed capacity consumption to reflect plant setup and a non-linear capacity-consumption function of the total product portfolio allocated to the plant. To develop the capacity consumption function the authors build product families with similar processing requirements and consider effects from intra- and inter-product family interactions. Based on a linear relaxation the authors explore both tabu-search heuristics and branch-and-bound algorithms to obtain solutions. 

When using a linear method, such as LDA, the underlying assumption is that the two classes are linearly separable. This, of course, is generally not true. If linear separability is not possible, then with enough samples, the more powerful quadratic discriminant analysis (QDA) works better, because it allows the hypersurface that separates the classes to be curved (quadratic). Unfortunately, the clinical reality of small-sized data sets denies us this choice. 

As illustrated in Figure 5, SVMs work by constructing a hyper-plane in a higher-dimensional feature space of the input data,91 and use the hyper-plane (represented as Hi and H2) to enforce a linear separation of input samples, which belong to different classes (represented as Class O and Class X). The samples that lie on boundaries of different classes are referred to as support vectors. The underlying principle behind SVM-based classification is to maximize the margin between the support vectors using kernel functions. In the case of... 

FIGURE 23.5 Schematic diagram showing the difference between linearly separable activity classes (a) and an embedded (non-linear) structure (b). For example, the active compounds (yellow) in (a) tend to have higher values of PCI and lower values of PC2 than the inactives (blue). While in (b), activity only occurs within a limited range of values of both PCI and PC2 and compounds outside this region are inactive. Data in (a) could be classified by LDA, while the data in (b) could be analyzed using SIMCA for example. 

Linear SVM Classifiers. When the data set is linearly separable, the decision function f x) = y(x) to separate the classes is given by ... 

However, in real life, many nonseparable (linear or nonlinear) classification problems occur, which practically means that distributions between two classes are overlapping. This implies that misclassihcations should be tolerated. Therefore, a set of slack variables > 0) is introduced in the margin minimization approach used for the linearly separable case, allowing some samples inside the margin. For this purpose. Equation 13.11 is replaced by Equation 13.12. 

---