Network two-layer

Neurons are not used alone, but in networks in which they constitute layers. In Fig. 33.21 a two-layer network is shown. In the first layer two neurons are linked each to two inputs, x, and X2- The upper one is the one we already described, the lower one has w, = 2, W2 = 1 and also 7= 1. It is easy to understand that for this neuron, the output )>2 is 1 on and above line b in Fig. 33.22a and 0 below it. The outputs of the neurons now serve as inputs to a third neuron, constituting a second layer. Both have weight 0.5 and 7 for this neuron is 0.75. The output yfi j, of this neuron is 1 if E = 0.5 y, + 0.5 y2 > 0.75 and 0 otherwise. Since y, and y2 have as possible values 0 and 1, the condition for 7 > 0.75 is fulfilled only when both are equal to 1, i.e. in the dashed area of Fig. 33.22b. The boundary obtained is now no longer straight, but consists of two pieces. This network is only a simple demonstration network. Real networks have many more nodes and transfer functions are usually non-linear and it will be intuitively clear that boundaries of a very complex nature can be developed. How to do this, and applications of supervised pattern recognition are described in detail in Chapter 44 but it should be stated here that excellent results can be obtained. 

Each set of mathematical operations in a neural network is called a layer, and the mathematical operations in each layer are called neurons. A simple layer neural network might take an unknown spectrum and pass it through a two-layer network where the first layer, called a hidden layer, computes a basis function from the distances of the unknown to each reference signature spectrum, and the second layer, called an output layer, that combines the basis functions into a final score for the unknown sample. 

The field of artificial neural networks is a new and rapidly growing field and, as such, is susceptible to problems with naming conventions. In this book, a perceptron is defined as a two-layer network of simple artificial neurons of the type described in Chapter 2. The term perceptron is sometimes used in the literature to refer to the artificial neurons themselves. Perceptrons have been around for decades (McCulloch Pitts, 1943) and were the basis of much theoretical and practical work, especially in the 1960s. Rosenblatt coined the term perceptron (Rosenblatt, 1958). Unfortunately little work was done with perceptrons for quite some time after it was realized that they could be used for only a restricted range of linearly separable problems (Minsky Papert, 1969). 

For our two-wavelength spectral data, a two-layer network is adequate to achieve the desired separation. A suitable neural network, with the weight vectors, is illustrated in Figure 18. 

The neural networks considered here consist of an input layer, which receives the input signals and in the simplest case is connected to a second layer, the output layer (two-layer network). Between the input and output layers, additional layers may be arranged. They are termed hidden layers (Figure 8.7). 

The concept of the autoassociative memory was extended to bidirectional associative memories (BAM) by Kosko (1987,1988). This memory, shown in Fig. 19.30, is able to associate pairs of the patterns a and b. This is the two-layer network with the output of the second layer connected directly to the input of the first layer. The weight matrix of the second layer is and W for the first layer. The rectangular weight matrix W is obtained as a sum of the cross-correlation matrixes... 

FIGURE 19.30 An example of the bi-directional autoassociative memory (a) drawn as a two-layer network with circulating signals, (b) drawn as two-layer network with bi directional signal flow. 

The radial basis function (RBF) network is a two layer network whose output nodes form a linear combination of non-linear basis functions computed by the hidden layer nodes [137-143]. The basis functions in the hidden layer produce a significant nonzero response only when the input falls within a small localized region of the input space (receptive field). In general, the hidden layer nodes use Gaussian response functions, with the position (w) and width (a) used as variables ... 

In spite of being actually partitioned into L+1 layers, a neural network with such an architecture is conventionally called an L-layer network (due to the fact that signals undergo transformations only in the layers of hidden and output neurons, not in the input layer). In particular, a one-layer network is a layered neural network without hidden neurons, whereas a two-layer network is a neural network in which only connections from input to hidden neurons and from hidden to output neurons are possible. 

The network architecture is indicated in the following way number of inputs—number of neurons in the first layer— number of neurons in the second layer e.g. 9-7-1 means that the two-layer network consists of nine inputs, seven neurons in the first layer and one neuron in the second layer (the number of neurons in the last layer equals the number of outputs). 

Once a design is known for the first two layers of the onion (i.e., reactors and separators only), the overall total cost of this design for all four layers of the onion (i.e., reactors, separators, heat exchanger network, and utilities) is simply the total cost of all reactors and separators (evaluated explicitly) plus the total cost target for heat exchanger network and utilities. 

Figure 9-16. ArtiFicial neural network architecture with a two-layer design, compri ing Input units, a so-called hidden layer, and an output layer. The squares Inclosing the ones depict the bias which Is an extra weight (see Ref, [10 for further details),...

A counter-propagation network is a method for supervised learning which can be used for prediction, It has a two-layer architecture where each netiron in the upper layer, the Kohonen layer, has a corresponding netiron in the lower layer, the output layer (sec Figure 9-21). A trained counter-propagation network can be used as a look-up tabic a neuron in one layer is used as a pointer to the other layer. 

These pairs of encoded structures and their (R spectra are used to ti ain a counterpropagation network (see Section 9.5.5). The two-layer netwoi k pi ocesses the structural information in its upper part and the spectral information in its lower part. Thus the network learns the correlation between the structures and their (R spec tra. This prnciedine is shown in Figine 10.2-8. 

Consider a three layer network. Let the input layer be layer one ( = 1), the hidden layer be layer two ( = 2) and the output layer be layer three ( = 3). The baek-propagation eommenees with layer three where dj is known and henee 8j ean be ealeulated using equation (10.69), and the weights adjusted using equation (10.71). To adjust the weights on the hidden layer = 2) equation (10.69) is replaeed by... 

Although the linear activation function passes more information from the input to a node to its output than a binary function does, it is of limited value in layered networks as two nodes in succession that both use a linear activation function are equivalent to a single node that employs the same function, thus adding an extra layer of nodes does not add to the power of the network. This limitation is removed by the use of curved activation functions. 

---