Kernels Radial basis function

Radial basis function networks (RBF) are a variant of three-layer feed forward networks (see Fig 44.18). They contain a pass-through input layer, a hidden layer and an output layer. A different approach for modelling the data is used. The transfer function in the hidden layer of RBF networks is called the kernel or basis function. For a detailed description the reader is referred to references [62,63]. Each node in the hidden unit contains thus such a kernel function. The main difference between the transfer function in MLF and the kernel function in RBF is that the latter (usually a Gaussian function) defines an ellipsoid in the input space. Whereas basically the MLF network divides the input space into regions via hyperplanes (see e.g. Figs. 44.12c and d), RBF networks divide the input space into hyperspheres by means of the kernel function with specified widths and centres. This can be compared with the density or potential methods in pattern recognition (see Section 33.2.5). 

An algorithm for computing the decision boundary thus requires the choice of the kernel function frequently chosen are radial basis functions (RBFs). A further input parameter is the priority of the size constraint for used in the optimization problem (Equation 5.38). This constraint is controlled by a parameter that is often denoted by y. A large value of y forces the size of to be small, which can lead to an overht and to a wiggly... 

Support Vector Machine (SVM) is a classification and regression method developed by Vapnik.30 In support vector regression (SVR), the input variables are first mapped into a higher dimensional feature space by the use of a kernel function, and then a linear model is constructed in this feature space. The kernel functions often used in SVM include linear, polynomial, radial basis function (RBF), and sigmoid function. The generalization performance of SVM depends on the selection of several internal parameters of the algorithm (C and e), the type of kernel, and the parameters of the kernel.31... 

SVM s are an outgrowth of kernel methods. In such methods, the data is transformed with a kernel equation (such as a radial basis function) and it is in this mathematical space that the model is built. Care is taken in the constmction of the kernel that it has a sufficiently high dimensionality that the data become linearly separable within it. A critical subset of transformed data points, the support vectors , are then used to specify a hyperplane called a large-margin discriminator that effectively serves as a hnear model within this non-hnear space. An introductory exploration of SVM s is provided by Cristianini and Shawe-Taylor and a thorough examination of their mathematical basis is presented by Scholkopf and Smola. ... 

The Analyze software uses the Kernel PLS method [114] with two key parameters, the number of latent variables and sigma. In this study these values were fixed at 5 and 10, respectively. K-PLS uses kernels and can therefore be seen as a nonlinear extension of the PLS method. The commonly used radial basis function kernel or Gaussian kernel was applied, where the kernel is expressed as [142]... 

These transformations are executed by using so-called kernel functions. The kernel functions can be both linear and nonlinear in nature. The most commonly used kernel function is of the latter type and called the radial basis function (RBF). There are a number of parameters, for example, cost functions and various kernel settings, within the SVM applications that will affect the statistical quality of the derived SVM models. Optimization of those variables may prove to be productive in deriving models with improved performance [97]. The original SVM protocol was designed to separate two classes but has later been extended to also handle multiple classes and continuous data [80]. 

This transformation into the higher-dimensional space is realized with a kernel function. The best function used depends on the initial data. In the SVM literature, typical kernel functions applied for classification are linear and polynomial kernels, or radial basis functions. Depending on the applied kernel function, some parameters must be optimized, for instance, the degree of the polynomial function (33,34). Once the data are transformed to another dimensional space by the kernel function, linear SVM can be applied. The main parameter to optimize with the SVM algorithm for nonseparable cases, as described in the previous section, is the regularization parameter, C. 

The Gaussian kernel is used in potential function classifiers, also known as radial basis function networks. A sigmoid kernel implements a multilayer perceptron (cf. Section 8.2) with a single hidden layer. 

For datasets that are not linearly separable, support vector machines map the data into higher dimensional space where the training set is separable via some transformation K x < (x). A kernel function K(Xi, x ) = (< (x,), < (x )) computes inner products in some expanded feature space. Some kernel functions such as linear K(Xj, x ) = and Gaussian (radial-basis function) K(Xi, Xj) = exp(— x, —x,jp/2a ) are widely used. 

The steps for building a radial basis function (RBF) kernel-based SVM model irsing LibSVM are enumerated here (Fig. 3.6). 

For SVM method, determinants for the kernel function and its parameters are the important steps in the method application. The Radial Basis Function (RBF) (Vapnik, 1995, Tax Duin 1999, Scholkopf et al., 1999) were employed for the kernel ... 

Considering the CS2 example, several assays were made considering four different kernels linear, polynomial (with different degrees), radial basis functions (RBF) and a sigmoid type. The RMSEC and the RMSEP errors for calibration and validation were considered in order to select a model and a satisfactory trade-off searched for. As expected, some good fits yielded nonuseful predictions for the unknowns. 

Using the SVM-BFS method, we select the features for the three artificial problems. The selected subset and its corresponding errors of each step are listed in Table 4.5. The SVM used in the feature selection of these three data sets are linear SVM, nonlinear SVM with radial basis function (RBF) kernel (cr = 1.54) and SVM with RBF kernel ([Pg.69]

Support vector classifiers with linear, polynomial and radial basis function (rbf) kernels are trained to classify the test data. For evaluating the performance of a classifier the following parameters are calculated ... 

The next two experiments were performed with the B spline kernel (Figure 6a) and the exponential radial basis function (RBF) kernel (Figure 6b). Both SVM models define elaborate hyperplanes, with a large number of support vectors (11 for spline, 14 for RBF). The SVM models obtained with the exponential RBF kernel acts almost like a look-up table, with all but one... 

Figure 6 SVM classification models for the dataset from Table 1 (a) B spline kernel, degree 1, Eq. [72] (b) exponential radial basis function kernel, a = 1, Eq. [67].

Figure 36 SVM classification models obtained with the Gaussian radial basis function kernel (Eq. [66]) for the dataset from Table 5 (a) a = 1 (b) a = 10.

Radial basis functions (RBF) are widely used kernels, usually in the Gaussian form ... 

Frequently the exclusive use of the RBF kernel is rationalized by mentioning that it is the best possible kernel for SVM models. The simple tests presented in this chapter (datasets from Tables 1-6) surest that other kernels might be more useful for particular problems. For a comparative evaluation, we review below several SVM classification models obtained with five important kernels (linear, polynomial, Gaussian radial basis function, neural, and anova) and show that the SVM prediction capability varies significantly with the kernel type and parameters values used and that, in many cases, a simple linear model is more predictive than nonlinear kernels. 

The table reports the experiment number Exp, capacity parameter C, kernel type K (linear L polynomial P radial basis function R neural N anova A), and corresponding parameters, calibration results (TP,., true positive in calibration FN, false negative in calibration TN,-, true negative in calibration FP, false positive in calibration SV number of support vectors in calibration AC calibration accuracy), and L20%O prediction results (TPp, true positive in prediction FNp, false negative in prediction TNp, true negative in prediction FPp, false positive in prediction SVp, average number of support vectors in prediction ACp, prediction accuracy). 

In this section, we compared the prediction capabilities of five kernels, namely linear, polynomial, Gaussian radial basis function, nemal, and anova. Several guidelines that might help the modeler obtain a predictive SVM model can be extracted from these results (1) It is important to compare the predictions of a large number of kernels and combinations of parameters (2) the linear kernel should be used as a reference to compare the results from nonlinear kernels (3) some datasets can be separated with a linear hyperplane in such instances, the use of a nonlinear kernel should be avoided and (4) when the relationships between input data and class attribution are nonlinear, RBF kernels do not necessarily give the optimum SVM classifier. 

LIBSVM, http //www.csie.ntu.edu.tw/ cjlin/libsvm/. LIBSVM (Library for Support Vector Machines) was developed by Chang and Lin and contains C-classification, v-classification, s-regression, and v-regression. Developed in C++ and Java, it also supports multiclass classification, weighted SVMs for unbalanced data, cross-validation, and automatic model selection. It has interfaces for Python, R, Spins, MATLAB, Perl, Ruby, and LabVIEW. Kernels available include linear, polynomial, radial basis function, and neural (tanh). 

---