Hyperplane

Discriminant emalysis is a supervised learning technique which uses classified dependent data. Here, the dependent data (y values) are not on a continuous scale but are divided into distinct classes. There are often just two classes (e.g. active/inactive soluble/not soluble yes/no), but more than two is also possible (e.g. high/medium/low 1/2/3/4). The simplest situation involves two variables and two classes, and the aim is to find a straight line that best separates the data into its classes (Figure 12.37). With more than two variables, the line becomes a hyperplane in the multidimensional variable space. Discriminant analysis is characterised by a discriminant function, which in the particular case of hnear discriminant analysis (the most popular variant) is written as a linear combination of the independent variables ... 

The n-fold procedure (n > 2) produces an n-dimensional hyperplane in n -b 1 space. Lest this seem unnecessarily abstract, we may regard the n x n slope matrix as the matrix establishing a calibration srrrface from which we may determine n unknowns Xi by making n independent measurements y . As a final generalization, it should be noted that the calibration surface need not be planar. It might, for example, be a curwed sruface that can be represented by a family of quadratic equations. 

A related idea is used in the Line Then Plane (LTP) algorithm where the constrained optimization is done in the hyperplane perpencheular to the interpolation line between the two end-points, rather than on a hypersphere. 

The Locally Updated Plane.s- (LUP) minimization is related to the chain method, but the relaxation is here done in the hyperplane perpendieular to the reaction coordinate, rather than along a line defined by the gradient. Furthermore, all the points are moved in each iteration, rather than one at a time. 

Method Path minimization Path maximization Hyperplane minimization Hypersphere minimization Global minimization Points moved... 

We have seen that the output neuron in a binary-threshold perceptron without hidden layers can only specify on which side of a particular hyperplane the input lies. Its decision region consists simply of a half-plane bounded by a hyperplane. If one hidden layer is added, however, the neurons in the hidden layer effectively take an intersection (i.e. a Boolean AND operation) of the half-planes formed by the input neurons and can thus form arbitrary (possible unbounded) convex regions. ... 

Consider a simple perceptron with N continuous-valued inputs and one binary (— 1) output value. In section 10.5.2 we saw how, in general, an A -dimensional input space is separated by an (N — l)-dimensional hyperplane into two distinct regions. All of the points lying on one side of the hyperplane yield the output -)-l all the points on the other side of the hyperplane yield -1. 

A graph of this function shows that it is not until the number of points n is some sizable fraction of 2( V + 1) that an (N - l)-dimensional hyperplane becomes over constrained by the requirement to correctly separate out (N + 1) or fewer points. In therefore turns out that the capacity of a simple perceptron is given by a rather simple expression if the number of output neurons is small and independent of N, then, as —> oo, the maximum number of input-output fact pairs that can be... 

In the original problem one usually has m < n. Thus, the vertices of the region of solution lie on the coordinate planes. This follows from the fact that, generally, in n dimensions, n hyperplanes each of dimension (n — 1) intersect at a point. The dual problem defines a polytope in m-dimensional space. In this case not all vertices need lie on the coordinate planes. 

Clearly for a vector ( ) to be transversal it is necessary and sufficient that at = 0. These transversal vectors lie in a hyperplane that is tangent to the light cone along ku. For a transversal vector one now verifies that... 

The procedure of constructing a grid in the plane domain we have described above can easily be generalized to the case of an arbitrary p-dimensional domain. A grid so constructed is a result of the intersection of hyperplanes (planes for p = 3 or straight lines for p = 2)... 

The transformation to this LP program is graphically depicted in Fig. 6 for the case when c is a scalar. For each data pair (x, y,) the term y, -represents two im + l)-dimensional hyperplanes, z =yi Lc dkiXi) and z = -y, + LckOkOti)- For the scalar case, these correspond... 

Radial basis function networks (RBF) are a variant of three-layer feed forward networks (see Fig 44.18). They contain a pass-through input layer, a hidden layer and an output layer. A different approach for modelling the data is used. The transfer function in the hidden layer of RBF networks is called the kernel or basis function. For a detailed description the reader is referred to references [62,63]. Each node in the hidden unit contains thus such a kernel function. The main difference between the transfer function in MLF and the kernel function in RBF is that the latter (usually a Gaussian function) defines an ellipsoid in the input space. Whereas basically the MLF network divides the input space into regions via hyperplanes (see e.g. Figs. 44.12c and d), RBF networks divide the input space into hyperspheres by means of the kernel function with specified widths and centres. This can be compared with the density or potential methods in pattern recognition (see Section 33.2.5). 

Methods based on linear projection exploit the linear relationship among inputs by projecting them on a linear hyperplane before applying the basis function (see Fig. 6a). Thus, the inputs are transformed in combination as a linear weighted sum to form the latent variables. Univariate input analysis is a special case of this category where the single variable is projected on itself. 

Partition-based methods address dimensionality by selecting input variables that are most relevant to efficient empirical modeling. The input space is partitioned by hyperplanes that are perpendicular to at least one of the input axes, as depicted in Fig. 6d. 

Of the several approaches that draw upon this general description, radial basis function networks (RBFNs) (Leonard and Kramer, 1991) are probably the best-known. RBFNs are similar in architecture to back propagation networks (BPNs) in that they consist of an input layer, a single hidden layer, and an output layer. The hidden layer makes use of Gaussian basis functions that result in inputs projected on a hypersphere instead of a hyperplane. RBFNs therefore generate spherical clusters in the input data space, as illustrated in Fig. 12. These clusters are generally referred to as receptive fields. 

Methods based on linear projection transform input data by projection on a linear hyperplane. Even though the projection is linear, these methods may result in either a linear or a nonlinear model depending on the nature of the basis functions. With reference to Eq. (6), the input-output model for this class of methods is represented as... 

Alternatively, methods based on nonlocal projection may be used for extracting meaningful latent variables and applying various statistical tests to identify kernels in the latent variable space. Figure 17 shows how projections of data on two hyperplanes can be used as features for interpretations based on kernel-based or local methods. Local methods do not permit arbitrary extrapolation owing to the localized nature of their activation functions. 

Because a hyperplane corresponds to a boundary between pattern classes, such a discriminant function naturally forms a decision rule. The global nature of this approach is apparent in Fig. 19. An infinitely long decision line is drawn based on the given data. Regardless of how closely or distantly related an arbitrary pattern is to the data used to generate the discriminant, the pattern will be classified as either o>i or <02. When the arbitrary pattern is far removed from the data used to generate the discriminant, the approach is extremely prone to extrapolation errors. 

Figure 20 shows more definitively how the location and orientation of a hyperplane is determined by the projection directions, a and the bias, o- Given a pattern vector x, its projection on the linear discriminant is in the a direction and the distance is calculated as d(x ) / cf The problem is the determination of the weight parameters for the hyper-plane ) that separate different pattern classes. These parameters are typically learned using labeled exemplar patterns for each of the pattern classes. 

This function provides a convenient means of determining the location of an arbitrary pattern x in the representation space. As shown, patterns above the hyperplane result in z(x) > 0, while patterns below generate values of z(x) < 0. The simplest form of the decision rule then is... 

The role of a boundary in a manifold with boundary can be interpreted with reference to a hyperplane within a Euclidean space E using the concept of halfspace, where the hyperplane is in fact the boundary of the half-space. By appropriate reordering of the coordinates, a half-space Hn becomes the subset of a Euclidean space En containing all points of En with non-negative value for the last coordinate. 

The well-known Box-Wilson optimization method (Box and Wilson [1951] Box [1954, 1957] Box and Draper [1969]) is based on a linear model (Fig. 5.6). For a selected start hyperplane, in the given case an area A0(xi,x2), described by a polynomial of first order, with the starting point yb, the gradient grad[y0] is estimated. Then one moves to the next area in direction of the steepest ascent (the gradient) by a step width of h, in general... 

Near the optimum both the step width and the model of the hyperplane are changed, the latter mostly from a first order model to a second order model. The vicinity of the optimum can be recognized by the coefficients fli,a2,... of Eq. (5.14) which approximate to zero or change their sign, respectively. For the second order model mostly a Box-Behnken design is used. 

The basis upon which this concept rests is the very fact that not all the data follows the same equation. Another way to express this is to note that an equation describes a line (or more generally, a plane or hyperplane if more than two dimensions are involved. In fact, anywhere in this discussion, when we talk about a calibration line, you should mentally add the phrase ... or plane, or hyperplane... ). Thus any point that fits the equation will fall exactly on the line. On the other hand, since the data points themselves do not fall on the line (recall that, by definition, the line is generated by applying some sort of [at this point undefined] averaging process), any given data point will not fall on the line described by the equation. The difference between these two points, the one on the line described by the equation and the one described by the data, is the error in the estimate of that data point by the equation. For each of the data points there is a corresponding point described by the equation, and therefore a corresponding error. The least square principle states that the sum of the squares of all these errors should have a minimum value and as we stated above, this will also provide the maximum likelihood equation. 

A calculation procedure could, in theory, predict at once the distribution of mass within a system and the equilibrium mineral assemblage. Brown and Skinner (1974) undertook such a calculation for petrologic systems. For an -component system, they calculated the shape of the free energy surface for each possible solid solution in a rock. They then raised an n -dimensional hyperplane upward, allowing it to rotate against the free energy surfaces. The hyperplane s resting position identified the stable minerals and their equilibrium compositions. Inevitably, the technique became known as the crane plane method. 

Let X be a smooth projective variety over k with an action of a torus H which has only finitely many fixed points. A one-parameter subgroup Gm —> H of H which does not lie in a finite set of given hyperplanes in the lattice of one-parameter groups of H will have the same fixed points as H. In future we call such a one-parameter group general . Thus the induced action of a general one-parameter group Gm — G has only finitely many fixed points on P. ... 

We again want to use the Porteous formula. Let H = OpN(l) be the hyperplane bundle on Pn- We will denote by the same letter its restriction to X and its first Chern class. 

The class of the locus, where X has nth order contact with a hyperplane in is... 

---