Pattern input

Feed-back models can be constructed and trained. In a constructed model, the weight matrix is created by adding the output product of every input pattern vector with itself or with an associated input. After construction, a partial or inaccurate input pattern can be presented to the network and, after a time, the network converges to one of the original input patterns. Hopfield and BAM are two well-known constructed feed-back models. 

Repeat this process for all input patterns. One iteration or epoch is defined as one weight correction for all examples of the training set. 

As described in Section 44.5.5, the weights are adapted along the gradient that minimizes the error in the training set, using the back-propagation strategy. One iteration is not sufficient to reach the minimum in the error surface. Care must be taken that the sequence of input patterns is randomized at each iteration, otherwise bias can be introduced. Several (50 to 5000) iterations are typically required to reach the minimum. 

Calculate a specified similarity measure, D, between each weight vector and a randomly chosen input pattern, x. 

ART networks consist of units that contain a weight vector of the same dimension as the input patterns. Each unit is meant to represent one class or cluster in the input patterns. The structure of the ART network is such that the number of units is larger than the expected number of classes. The units in excess are dummy units that can be taken into use when a new input pattern shows up that does not belong to any of the already learned classes. 

The similarity between x, and the winning unit is compared with a threshold value, p, in the range from zero to one. When p, < P the input pattern, x, is not considered to fall into the existing class. It is decided that a so-called novelty is detected and the input vector is copied into one of the unused dummy units. Otherwise the input pattern, x, is considered to fall into the existing class (to resonate with it). A large p will result in many novelties, thus many small clusters. A small p results in few novelties and thus in a few large clusters. 

When the resonance step succeeds the weight vector of the winning unit is changed. It adapts itself a little towards the new input pattern x, belonging to the same class, according to ... 

Usually the learning rate, T), is chosen between 0 and 1. In this step the network incorporates the new information present in the input object by moving the centroid of the class a little towards the new input pattern x,. This step is intended to make the network flexible enough when clusters are changing in time. 

Clustering lacks the strict Bayesian requirement that input patterns be identifiable with a known prototype for which underlying joint distributions or other statistical information is known. Rather, the clustering approach... 

This rule can be easily extended so that the classification of the majority of k nearest neighbors is used to assign the pattern class. This extension is called the k-nearest neighbor rule and represents the generalization of the 1-NN rule. Implementation of the k-NN rule is unwieldy at best because it requires calculation of the proximity indices between the input pattern x and all patterns z , of known classification. 

Carpenter, G. A., and Grossberg, S., ART2 self-organization of stable category recognition codes for analog input patterns, Appl. Opt. 26, 4919 (1987a). 

Backpropagation has two phases. In the first, an input pattern is presented to the network and signals move through the entire network from its inputs to its outputs so the network can calculate its output. In the second phase, the error signal, which is a measure of the difference between the target response and actual response, is fed backward through the network, from the outputs to the inputs and, as this is done, the connection weights are updated. 

Select a random input pattern, with its corresponding target output. 

Through a process of training, the weights evolve so that each node forms a prototypical blend of groups of input patterns. Just as with the clustering of animals, scientists of similar characteristics will be positioned close together on the map. 

If the input pattern and the weights vectors for the nodes in a four-node map were... 

The input pattern is compared with the weights vector at every node to determine which set of node weights it most strongly resembles. In this example, the height, hair length, and waistline of each sample pattern will be compared with the equivalent entries in each node weight vector. 

Since the node weights are initially seeded with random values, at the start of training no node is likely to be much like the input pattern. Although the match between pattern and weights vectors will be poor at this stage, determination of the winning node is simply a competition among nodes and the absolute quality of the match is unimportant. 

Create a SOM in which each input pattern equals the factors of an... 

Initially a map of minimal size is prepared that consists of as few as three or four nodes. Since the map at this stage is so small, it is very quick to train. As training continues and examples of different classes are discovered in the database, the map spreads itself out by inserting new nodes to provide the extra flexibility that will be needed to accommodate these classes. The map continues to expand until it reaches a size that offers an acceptable degree of separation of samples among the different classes. As in a SOM, on the finished map, input patterns that are similar to one another should be mapped... 

---