Weights vectors

An initial weight vector with unit length and components 

---

The weights w, have to be coded as a vector to enable the electronic processing This is done by gathering all the weights of one neuron in a weight vector (Figure 9-14),... 

This is done by calculating the Euclidean distance between the input data vector Xc and the weight vectors Wj of all neurons ... 

The gradient can be used to optimize the weight vector according to the method of steepest-descent ... 

Each agent is equipped with a user-specified personality - or internal value system defined by a six-component personality weight vector, m = wi,W2,. , wq),... 

In NIPALS one starts with an initial vector t with n arbitrarily chosen values (Fig. 31.12). In a first step, the matrix product of the transpose of the nxp table X with the n-vector t is formed, producing the p elements of vector w. Note that in the traditional NIPALS notation, w has a different meaning than that of a weighting vector which has been used in Section 31.3.6. In a second step, the elements of the p-vector w are normalized to unit sum of squares This prevents values from becoming too small or too large for the purpose of numerical computation. The... 

So fiir, we have described only situations with two classes. The method can also be applied to K classes. It is then sometimes called descriptive linear discriminant analysis. In this case the weight vectors can be shown to be the eigenvectors of the matrix ... 

Another approach to eq. (44.2) is to add an extra dimension to the object vector X, on which all objects have the same value. Usually 1 is taken for this extra term. The 0 term can then be included in the weight vector, w (w,W2,-0). This is the same procedure as for MLR where an extra column of ones is added to the X-matrix to accommodate for the intercept (Chapter 10). The objects are then characterized by the vector xj (X, X2 1). Equation 44.2 can then be written ... 

Initialize the weight vectors of all units with random values. 

Calculate a specified similarity measure, D, between each weight vector and a randomly chosen input pattern, x. 

After the training procedure, the weight vectors of the units are fixed and the map is ready to be interpreted. There are different possibilities to interpret the weight combination, depending on the purpose of the network use. In this section some possibilities are described. 

The output-activity map. A trained Kohonen network yields for a given input object, X, one winning unit, whose weight vector, is closest (as defined by the criterion used in the learning procedure) to x,. However, X may be close to the weight vectors, w, of other units as well. The output yj of the units of the map can also be defined as ... 

D is the similarity measure as used in the training procedure. This results in a map as in Fig.44.26a. This map allows the inspection of regions (neighbouring neurons) that have a similar weight vector as a given input x,. Note that each input x, yields a different output activity map. 

Due to the Kohonen learning algorithm, the individual weight vectors in the Kohonen map are arranged and oriented in such a way that the structure of the input space, i.e. the topology is preserved as well as possible in the resulting... 

ART networks consist of units that contain a weight vector of the same dimension as the input patterns. Each unit is meant to represent one class or cluster in the input patterns. The structure of the ART network is such that the number of units is larger than the expected number of classes. The units in excess are dummy units that can be taken into use when a new input pattern shows up that does not belong to any of the already learned classes. 

Initialize the weights of all units (in a (pxc) matrix W) with a fixed value. The parameter p is the length of the weight vector and c is the total number of units. Usually the fixed value is used for the initial weights, such that the length of the weight vector is scaled to unity. 

The first input vector is copied into the weight vector of the first unit, which becomes now an active unit. 

For the next input vector the similarity, p,, with the weight vector, W , of each active unit, k is calculated ... 

When the resonance step succeeds the weight vector of the winning unit is changed. It adapts itself a little towards the new input pattern x, belonging to the same class, according to ... 

Not only are the lengths of the pattern and weights vectors identical, the individual entries in them share the same interpretation. 

Set the learning rate, T, to a small positive value 0 < r < 1. Fill the weights vector at each node with random numbers. 

Calculate how similar the sample pattern is to the weights vector at each node in turn, by determining the Euclidean distance between the sample pattern and the weights vector. 

Select the winning node, which is the node whose weights vector most strongly resembles the sample pattern. 

Update the weights vector at the winning node to make it slightly more like the sample pattern. 

Update the weights vectors of nodes in the neighborhood of the winning node. 

Now that the SOM has been constructed and the weights vectors have been filled with random numbers, the next step is to feed in sample patterns. The SOM is shown every sample in the database, one at a time, so that it can learn the features that characterize the data. The precise order in which samples are presented is of no consequence, but the order of presentation is randomized at the start of each cycle to avoid the possibility that the map may learn something about the order in which samples appear as well as the features within the samples themselves. A sample pattern is picked at random and fed into the network unlike the patterns that are used to train a feedforward network, there is no target response, so the entire pattern is used as input to the SOM. 

Each value in the chosen sample pattern is compared in turn with the corresponding weight at the first node to determine how well the pattern and weights vector match (Figure 3.9). A numerical measure of the quality of the match is essential, so the difference between the two vectors, dpq, generally defined as the squared Euclidean distance between the two, is calculated ... 

Both the sample vector and the node vector contain n entries x is the -th entry in the pattern vector for sample q, while a is the -th entry in the weights vector at node p. This comparison of pattern and node weights is made for each node in turn across the entire map. 

Example 6 Calculating the Distances between Sample and Weights Vector... 

If the input pattern and the weights vectors for the nodes in a four-node map were... 

The input pattern is compared with the weights vector at every node to determine which set of node weights it most strongly resembles. In this example, the height, hair length, and waistline of each sample pattern will be compared with the equivalent entries in each node weight vector. 

Since the node weights are initially seeded with random values, at the start of training no node is likely to be much like the input pattern. Although the match between pattern and weights vectors will be poor at this stage, determination of the winning node is simply a competition among nodes and the absolute quality of the match is unimportant. 

---