Function kernel

In Eq (25), the integration kernel function K x)= rr x is smooth everywhere except the singularity point (x = 0). In a numerical analysis, the integration has to be evaluated in discrete form over a grid with the mesh size h... 

The problem is, however, the evaluation (36) may create a considerable error due to the singularity of the kernel function. In these cases, a correction process has to be applied. Here a two-level grid system with H=2h is used as an example to illustrate the approach of correction. 

Radial basis function networks (RBF) are a variant of three-layer feed forward networks (see Fig 44.18). They contain a pass-through input layer, a hidden layer and an output layer. A different approach for modelling the data is used. The transfer function in the hidden layer of RBF networks is called the kernel or basis function. For a detailed description the reader is referred to references [62,63]. Each node in the hidden unit contains thus such a kernel function. The main difference between the transfer function in MLF and the kernel function in RBF is that the latter (usually a Gaussian function) defines an ellipsoid in the input space. Whereas basically the MLF network divides the input space into regions via hyperplanes (see e.g. Figs. 44.12c and d), RBF networks divide the input space into hyperspheres by means of the kernel function with specified widths and centres. This can be compared with the density or potential methods in pattern recognition (see Section 33.2.5). 

The output of a hidden unit, in case of a Gaussian kernel function is defined as ... 

The output of these hidden nodes, o, is then forwarded to all output nodes through weighted connections. The output yj of these nodes consists of a linear combination of the kernel functions ... 

The centroids represent the positions of Gauss kernels and are in this example positioned in the same place as the objects in the input space. The width factors do not change during training. The weights of each kernel function are obtained by training. 

In order to use the notional particles to estimate f x, we need a method to identify a finite sample of notional particles in the neighborhood of x on which to base our estimate. In transported PDF codes, this can be done by introducing a kernel function hw(s) centered at 5 = 0 with bandwidth W. For example, a so-called constant kernel function (Wand and lones 1995) can be employed ... 

For simplicity, we will let W = 1/(2M), i.e the bandwidth is inversely proportional to the number of grid cells. Another alternative is to use the M grid cells to define the kernel function ... 

Note that since the cell size is equal to 2W, the bandwidth for / , is the same as for hw. Other widely used kernel functions are described in Chapter 7. 

Using the kernel function, we can define the number of notional particles at point x by... 

Using the spatial grid kernel function, (6.207), the estimated histogram is given by... 

One of the simplest estimation techniques is to use a kernel function hw(x) (see, for example, (6.206), p. 301). However, care must be taken in choosing the form of the kernel function in order to ensure that desirable physical constraints are not violated. For example, with unit-weight particles, the requirement that the mixing model in (7.28)... 

In general, very few kernel functions will satisfy this condition. For example, it is easily shown that the grid-cell kernel... 

The kernel K(x,x — y) must satisfy this constraint for any integrable function (p. This is just the definition of the Dirac delta function K(x, x — y) = S(x — y). Note that, in this limit, the kernel function is equivalent to the filter function used in LES. As is well known in LES, filtering a function twice leads to different results unless the integral condition given above is satisfied. 

The kernel function could also be made simpler by using... 

The critical step of the inverse problem is a projection of the unknown function onto the base formed by the kernel functions. This base is not orthogonal and, because the number of kernel functions is finite, cannot describe perfectly the unknown function. For that reason, we write that the unknown function x /SC x) is the sum of its projections with component u, along the j th kernel functions plus any orthogonal complement labeled x fiC x) and belonging to the null-space of the kernel functions... 

Here, K x, tiij, s ) are the kernel functions with prototypes m and scale parameters sr For example, if the kernel function is the standard normal density function [Pg.183]

FIGURE 4.28 Visualization of kernel regression with two Gaussian kernels. The point sizes reflect the influence on the regression model. The point sizes in the left plot are for the solid kernel function, those in the right plot are for the dashed kernel function. 

There are various procedures to estimate the unknown regression parameters and the parameters for the kernel functions. One approach is to estimate prototypes ntj and scale parameters Sj separately by clustering methods, and then to estimate the regression parameters, however, this approach does not incorporate information of the y-variable. Another approach is to use optimization techniques to minimize the RSS for the residuals y, - fix,) obtained via Equation 4.95, for i = 1,..., n. 

An algorithm for computing the decision boundary thus requires the choice of the kernel function frequently chosen are radial basis functions (RBFs). A further input parameter is the priority of the size constraint for used in the optimization problem (Equation 5.38). This constraint is controlled by a parameter that is often denoted by y. A large value of y forces the size of to be small, which can lead to an overht and to a wiggly... 

In the following example, the effect of these parameter choices will be demonstrated. We use the same example as in Section 5.5 with two overlapping groups. Figure 5.20 shows the resulting decision boundaries for different kernel functions... 

The most important parameter choices for SVMs (Section 5.6) are the specification of the kernel function and the parameter y controlling the priority of the size constraint of the slack variables (see Section 5.6). We selected RBFs for the kernel because they are fast to compute. Figure 5.27 shows the misclassification errors for varying values of y by using the evaluation scheme described above for k-NN classification. The choice of y = 0.1 is optimal, and it leads to a test error of 0.34. 

Froehlich, H., Wegner, J.K., Sieker, F. and Zell, A. (2006) Kernel functions for attributed molecular graphs - a new similarity-based approach to ADME prediction in classification and regression. QSAR Combinatorial Science, 25, 317-326. 

---