Interpretation by Statistical Learning Therory

Section 2.1 has attempted to determine the maximal margin hyperplane in an intuitive way. Support Vector Machine, the successful implementation of statistical learning theory (SLT), are built on the basis of the maximal margin hyperplane described above. It is important to reveal the relationship between the formula (2.16) and SLT. 

Given a set of functions, the relationship between the empirical risk and the actual risk for this set of functions is one of most important research directions, which is known as the bounds on generalization ability of learning machines. As for binary classification problems, the following basic bounds describing the generalization ability of a threshold real-valued function (also known as indicator function) that minimize the empirical risk functional [131]  

The inequality (2.17) shows that the actual risk of the learning machine consists of two parts the first term on the right hand side of the inequality is the empirical risk (corresponding to the training errors) and the second term is called the VC confidence , which depends on the VC dimension of the set of fimctions (h) and the number of sample points ( ). Obviously, the VC confidence is a monotonic increasing function of h, which is true for any value of . 

According to Theorem 2.2, given some selection of learning machines whose empirical risk is zero, one wants to choose that learning machine whose associated set of functions has minimal VC dimension. At present, for the y-margin separating hyperplane, we quote an important theorem without proof as follows. For more details, see [132]. 

Theorem 2.3 Let vectors x e belong to a sphere of radius R. Then the set of -margin separating hyperplanes has the VC dimension h bounded by the inequality 

---