Classifier binary

FIGURE 3. A decision plane (a straight line in this 2-dimensional example) separates the two classes of objects and is defined by a decision vector w. 

The decision plane is usually defined by a decision vector (weight vector) w orthogonal to the plane and through the origin. The weight vector is very suitable to decide whether a point lies on the left or 

For more than two dimensions another equivalent equation for the scalar product is more convenient  

The development of a decision vector is usually computer time consuming- But the application of a given decision vector to a concrete classification requires only some multiplications and summations which can be easily done even by a pocket calculator- 

FIGURE 6. No decision plane through the origin is possible. 

---

In the following, we assume that all objects come from two groups (denoted as 1 and 2), and that they are classified to one of the two groups. We thus have a binary classifier without rejections. The goal is to derive a measure for the prediction performance of the classifier. This concept can be easily extended to the multiple group case. 

Freidlin and Simon (10) have shown, however, how one pivotal trial can be used potentially for both purposes—if the set of patients used to develop the classifier is kept distinct from the set of patients used to evaluate treatment benefit. Generally, however, the studies should be kept separate. Developmental studies are exploratory, though they should result in completely specified binary classifiers. Studies on which claims of drug benefit are based should be non-exploratory, but should instead test prospectively defined hypotheses about treatment effect in a pre-defined patient population. 

The Receiver Operator Characteristic curve (ROC curve) is a graphical plot of the sensitivity Sn versus false positive rate FPR for a binary classifier system as its discrimination threshold is varied. The ROC curve can also be represented equivalently by plotting the fraction of true positives (TP) versus the fraction of false positives (FP) (Figure C3). ROC analysis provides tools to select possibly optimal classification models. 

Fr h x) y). One common classification problem is binary classification where an example is placed in one of two mutually exclusive groups, y e 1, —1. Note that many binary classifiers produce a real-valued output such that R h0c). A threshold is applied to the real value, signj[ti(x)] e 1,-1 such that if the real value exceeds the threshold t then the output is 1 otherwise the output is —1. 

The performance of a classifier may be measured by a number of metrics. We have four basic counts to tabulate a binary prediction TP (True Positive), FP (False Positive), TN (True Negative), and FN (False Negative). Most metrics are calculated from these four numbers. A standard classifier minimizes the error estimated by the number of mistakes over the number of predictions this is often measured by the accuracy (TP + TN) /(TP + TN + FP + FN). Nevertheless, a binary classifier can make two types of errors, one for each class. For the positive class it is called sensitivity TP/(TP+FP). Similarly, the for the negative class it is called specificity TN/ TN + FN). Note that each of these metrics depends on the threshold used for a real-valued classifier, e.g. a higher threshold will lower the sensitivity and increase the specificity. 

Additional disadvantages are the simple class boundaries, the danger of wrong assignments of outliers, or the slow convergence. In addition, LLM is restricted to the separation of only two classes (binary classifier). 

In contrast to OVR, OVO method trains 2 = N N —1)/2 binary classifier models considering eaeh possible pair for a N class problem. Fig. 3 illustrates how six binary classifiers are trained in OVO method for a four class problem. The testing of a sample in OVO method is performed by max win strategy [33]. By this technique each of the trained binary classifier can deliver one vote for its favored class and the class with maximum votes specifies the class label of the sample. Thus as the number of classes increases the training and testing time also increase in this method. 

Fig. 2 One-versus-rest method of decomposition of a multiclass problem consisting of four different classes. For a four class problem, four binary classifiers tire trained...

NPPC [22] is a binary classifier and it classifies a pattern by the proximity of a test pattern to one of the two planes as shown in Fig. 5. The two planes are obtained by solving two nonlinear programming problems (NPP) with a quadratic form of loss function. Each plane is clustered around a particular class of data by minimizing sum squared distances of patterns from it and considering the patterns of the others class at a distance of 1 with soft errors. Thus, the objective of NPPC is to find two hyperplanes ... 

The extensive application of binary SVM classifier provoked several researchers to discover the efficient way of extending binary classifier to multi-class classifier [23-26], In the literature there are one-versus-rest (OVR), one-versus-one (OVO), directed acyclic graph (DAG), all at once, error correcting code etc. Among these we have used three most commonly used methods of multi-class SVM classifier in this application. A brief description of the OVR, OVO and DAG method is explained below. 

In OVR method k numbers of binary classifier models are constructed for a k class problem [23]. In doing so the patterns of ith class considered as the positive samples and the patterns of all other classes are considered as the negative samples for the ith binary classifier model. This decomposition method has been shown in Fig. 1 for a three class problem which shows that it builds three classifiers (a) class 1 versus classes 2 and 3, (b) class 2 versus classes 1 and 3, and (c) class 3 versus classes 1 and 2. The combined OVR decision function chooses the class of a sample that corresponds to the maximum value of the argument of k binary decision functions specified by the furthest positive hyperplane. 

FIGURE 8. Procedure for training and evaluation of a binary classifier. 

A less pretentious synonym for this method is "classification by adapt i ve (linear, binary) classifiers". ... 

The aim of regression analysis is to determine the statistical relation between pattern vectors Xj and a scope value z which is used to classify a pattern- A function s = s(x) that approximates z as closely as possible is therefore sought. For a binary classifier a scope value (forcing value) of z = +1 can be demanded for all pattern vectors of class 1 and a value of z = -1 for class 2. 

FIGURE 29. Probability density functions for a binary classifier with a continuous response s g and 92 probability densities for... 

In this method each binary classifier is trained to classify patterns into one of two classes separated by a cutoff point. Class 1 means... 

TABLE 4. Parallel arrangement of binary classifiers for the prediction of the number of atoms of a certain element in the molecule. 

In this method/ the binary classifiers are arranged in a branching network. Each classifier is trained to dichotomize a set of pattern vectors according to the scheme in Figure 30. For the training/ only those patterns pertinent to a distinct branch point should be used. The method suffers from the accumulation of errors for successive decisions. 

The number of the class 02) is coded as a binary number. For each binary digit/ a binary classifier is trained that predicts "zero" or "one". This classification method requires the smallest number of binary classifiers C85/ 1783. For example/ up to 8 classes can be discriminated by 3 binary classifiers (Table 5). The accuracy of the method can be improved by introducing additional binary digits (additional binary classifiers) to form an error-correcting code analogous to a "parity bit". 

---