SVM for the Classification of Linearly Non-Separable Data

In this section, we consider a training set T ol m patterns together with their classes, T= (xi,yi), (X2,y2), , m,ym) that can be separated 

In the previous section, we found that the OSH defined by a pair (w, b) is a buffer between class -1-1 and class —1 of patterns, with the property that it has the largest margin. The border toward the class -1-1 is defined by the hyperplane w x. + b = —1, whereas the border toward the class —1 is defined by the hyperplane w -x + b = —1. For the OSH, all class -1-1 patterns satisfy W X -b -1-1, whereas all class —1 patterns satisfy w x + b —1, and the learning set is classified without errors. 

Similarly, for a pattern (x,-, y,) from the class —1, the slack variable is defined as 

When slack variables are introduced to penalize misclassified patterns or patterns situated in the buffer region between FI and the corresponding border hyperplanes (FIi or FI2), the constraints imposed to the objective function are as follows  

The identification of an OSFI is much more difficult when slack variables are used, because the optimum classifier is a compromise between two opposing conditions. On the one hand, a good SVMC corresponds to a hyperplane 

---