Kernel trick

This paper is organized as follows. Sec. 2 provides the background to the kernel-based learning methods and kernel trick. Sec. 3 introduces a linear matched subspace and its kernel version. The orthogonal subspace detector is defined in Sec. 4 as well as its kernel version. In Sec. 5 we describe the conventional spectral matched filter ad its kernel version in the feature space and reformulate the expression in terms of the kernel function using the kernel trick. Finally, in Sec. 6 the adaptive subspace detector and its kernel version are introduced. Performance comparison between the conventional and the kernel versions of these algorithms is provided in Sec. 7 and conclusions are given in Sec. 8. 

Thus, using the kernel trick, the Lagrangian dual problem in the feature space will maximize the following problem ... 

The above decision function can be expressed in terms of kernel trick as follows fix) = sgn[y]yiaik(Xi,x) + b] (8)... 

The above formulation is known as linear NPPC. When the patterns are not linearly separable then one can use nonlinear NPPC. The linear NPPC can be extended to nonlinear classifiers by applying the kernel trick [18]. For nonlinearly separable patterns, the input data is first mapped into a higher dimensional feature space by some kernel function. In the feature space it implements a linear classifier which correspond a nonlinear separating surface in the input space. To apply this transformation, let k(.,.) be any nonlinear kernel function and define the augmented matrix ... 

Thus, the SVM is a hnear method in a high-dimensional feature space, which is nonhnearly related to tlie input space. Though the hnear algorithm works in the high-dimensional feature space, in practice it does not involve any computations in that space, since through the usage of the kernel trick all necessary computations are performed directly in the input space [23]. 

Where the x s are the two vectors of experimental data for samples 1 and 2, ( ) is the transpose of a vector, and a and b are constants. An appealing property of SVMs is that the a priori complex step of non-linear mapping can be calculated in the original space by the kernel functions after some key parameters are optimised. This means that the new dimensions arise from combinations of experimental variables. Another curious property is that the kernel itself yields a measure of the similarity between two samples in the feature space, using just the original data This is called the "kernel trick. Further, it is not necessary for the analyst to know the mathematical functions behind the kernel in advance. Once the type of kernel is selected e.g. linear, RBF or polynomial) the non-linear mapping functions will be set automatically. ... 

There are, however, some valuable tricks to play to ease these calculations. Again they depend on factorising the scheme into a kernel and a factors. 

It is Vapnik and his coworkers who found the trick of using kernel function as a tool to convert nonlinear data processing problems into linearly solvable problems in about 25 years later after Aizermann s work about the potential function method. 

---