Mercer kernel

In order to learn nonlinear relationships with a linear machine, we can apply a fixed nonlinear mapping of the data in input space to a feature space, and that the decision function is 

As we know, one important property of the linear learning machine is that they can be expressed in a dual representation. It means that the hypothesis function can be expressed as a linear combination of the training points, so that the decision rule can be evaluated using just inner products between the test point and the training points (see equation (2.24) and (2.53)). 

If we have a way of computing the inner product )- (x) in feature space directly as a function of the original input points, it becomes possible to merge the two steps to build a nonlinear learning machine. Such a direct computation method is called kernel function method. 

As one does not express the feature vectors explicitly, the number of operations required to compute the inner product is not necessarily proportional to the number of features by employing the kernel function. The use of kernel makes it possible to map the data implicitly into a feature space and to train a linear machine in such a space, the only information used about the training samples is the kernel matrix, and the key is to find a kernel function that can be evaluated efficiently. Once we have such a function, the decision rule can be evaluated by the computation of the kernel. In accordance with (2.21) to (2.33)  

The use of the kernel function is an attractive computational short-cut. A curious fact about using a kernel is that we do not need to know the underlying feature map which can learn in the feature space. In practice the approach taken is to define a kernel function directly, hence implicitly to define the feature space. In this way, we avoid the feature space not only in the computation of inner product, but also in the design of the learning machine itself. 

---

Let us observe this theorem, the positivity condition I K(x,z)/(x)/(z)dxdz > 0, V/ 6 Z2 (X), corresponds to the positive semi-definite condition in the finite case, this gives the second characterization of a kernel function that is proved to be most useful when we come to constructing kernels. We use the term kernel to refer to function satisfying this property, but in the literature these are often called Mercer kernel. 

Kernels are symmetric functions. Mercer s theorem provides a characterization of kernels a symmetric function K u, v) is a kernel function if and only if the matrix... 

In both the dual solution and decision function, only the inner product in the attribute space and the kernel function based on attributes appear, but not the elements of the very high dimensional feature space. The constraints in the dual solution imply that only the attributes closest to the hyperplane, the so-called SVs, are involved in the expressions for weights w. Data points that are not SVs have no influence and slight variations in them (for example caused by noise) will not affect the solution, provides a more quantitative leverage against noise in data that may prevent linear separation in feature space [42]. Imposing the requirement that the kernel satisfies Mercer s conditions (K(xj, must be positive semi-definite)... 

Now there is a further problem about the kernel function, i.e. given a function K, how to verify that it is a kernel. The answer is given by Mercer s condition [132], which will be discussed in next chapter. 

Then what is the sufficient condition for a function to be a kernel function The theoretical derivation can prove that if a function satisfies Mercer condition, it can be regard as a kernel function. 

The key to verifying a new symmetric function as a kernel is the condition stated in Mercer Theorem, that is, the requirement that the... 

If the kernel fC is a symmetric positive definite function, which satisfies the Mercer s conditions... 

---