Layered Networks

Consider a three layer network. Let the input layer be layer one ( = 1), the hidden layer be layer two ( = 2) and the output layer be layer three ( = 3). The baek-propagation eommenees with layer three where dj is known and henee 8j ean be ealeulated using equation (10.69), and the weights adjusted using equation (10.71). To adjust the weights on the hidden layer = 2) equation (10.69) is replaeed by... 

Carbon black is reinforced in polymer and mbber engineering as filler since many decades. Automotive and tmck tires are the best examples of exploitation of carbon black in mbber components. Wu and Wang [28] studied that the interaction between carbon black and mbber macromolecules is better than that of nanoclay and mbber macromolecules, the bound mbber content of SBR-clay nanocompound with 30 phr is still of high interest. This could be ascribed to the huge surface area of clay dispersed at nanometer level and the largest aspect ratio of silicate layers, which result in the increased silicate layer networking [29-32]. 

Neurons are not used alone, but in networks in which they constitute layers. In Fig. 33.21 a two-layer network is shown. In the first layer two neurons are linked each to two inputs, x, and X2- The upper one is the one we already described, the lower one has w, = 2, W2 = 1 and also 7= 1. It is easy to understand that for this neuron, the output )>2 is 1 on and above line b in Fig. 33.22a and 0 below it. The outputs of the neurons now serve as inputs to a third neuron, constituting a second layer. Both have weight 0.5 and 7 for this neuron is 0.75. The output yfi j, of this neuron is 1 if E = 0.5 y, + 0.5 y2 > 0.75 and 0 otherwise. Since y, and y2 have as possible values 0 and 1, the condition for 7 > 0.75 is fulfilled only when both are equal to 1, i.e. in the dashed area of Fig. 33.22b. The boundary obtained is now no longer straight, but consists of two pieces. This network is only a simple demonstration network. Real networks have many more nodes and transfer functions are usually non-linear and it will be intuitively clear that boundaries of a very complex nature can be developed. How to do this, and applications of supervised pattern recognition are described in detail in Chapter 44 but it should be stated here that excellent results can be obtained. 

The general stmcture is shown in Fig. 44.9. The units are ordered in layers. There are three types of layers the input layer, the output layer and the hidden layer(s). All units from one layer are connected to all units of the following layer. The network receives the input signals through the input layer. Information is then passed to the hidden layer(s) and finally to the output layer that produces the response of the network. There may be zero, one or more hidden layers. Networks with one hidden layer make up the vast majority of the networks. The number of units in the input layer is determined by p, the number of variables in the (nxp) matrix X. The number of units in the output layer is determined by q, the number of variables in the inxq) matrix Y, the solution pattern. 

Independent studies (Cybenko, 1988 Homik et al., 1989) have proven that a three-layered back propagation network will exist that can implement any arbitrarily complex real-valued mapping. The issue is determining the number of nodes in the three-layer network to produce a mapping with a specified accuracy. In practice, the number of nodes in the hidden layer are determined empirically by cross-validation with testing data. 

PPR is a linear projection-based method with nonlinear basis functions and can be described with the same three-layer network representation as a BPN (see Fig. 16). Originally proposed by Friedman and Stuetzle (1981), it is a nonlinear multivariate statistical technique suitable for analyzing high-dimensional data, Again, the general input-output relationship is again given by Eq. (22). In PPR, the basis functions 9m can adapt their shape to provide the best fit to the available data. 

Each set of mathematical operations in a neural network is called a layer, and the mathematical operations in each layer are called neurons. A simple layer neural network might take an unknown spectrum and pass it through a two-layer network where the first layer, called a hidden layer, computes a basis function from the distances of the unknown to each reference signature spectrum, and the second layer, called an output layer, that combines the basis functions into a final score for the unknown sample. 

One layer of input nodes and another of output nodes form the bookends to one or more layers of hidden nodes Signals flow from the input layer to the hidden nodes, where they are processed, and then on to the output nodes, which feed the response of the network out to the user. There are no recursive links in the network that could feed signals from a "later" node to an "earlier" one or return the output from a node to itself. Because the messages in this type of layered network move only in the forward direction when input data are processed, this is known as a feedforward network. 

Although the linear activation function passes more information from the input to a node to its output than a binary function does, it is of limited value in layered networks as two nodes in succession that both use a linear activation function are equivalent to a single node that employs the same function, thus adding an extra layer of nodes does not add to the power of the network. This limitation is removed by the use of curved activation functions. 

Finally, as another simple example of description (and symbolic representation) of structures in terms of layer stacking sequence, we now examine structures which can be considered as generated by layer networks containing squares. A typical case will be that of structures containing 44 nets of atoms (Square net S net). The description of the structures will be made in terms of the separation of the different nets, along the direction perpendicular to their plane, and of the origin and orientation of the unit cell. 

An open-tubular column is a capillary bonded with a wall-supported stationary phase that can be a coated polymer, bonded molecular monolayer, or a synthesized porous layer network. The inner diameters of open-tubular CEC columns should be less than 25 pm that is less than the inner diameters of packed columns. The surface area of fused silica tubing is much less than that of porous packing materials. As a result, the phase ratio and, hence, the sample capacity for open-tubular columns are much less than those for packed columns. The small sample capacity makes it difficult to detect trace analytes. 

Fig. 14. A portion of the layer network of OnlGeCt CHaCC H). The alkyl carboxylate groups, which have been omitted for clarity, alternate above and below the polymeric sheet.

Fig. 12 Noncovalent synthesis of a layered network with large cavity based on hydrogen bonding between trithiocyanuric acid and bipyridine. Three-dimensional channels with benzene molecules can be seen [from Pedireddi et al. (reproduced with permission from ref. 30(6))].

Fig. 3 The two-dimensional layered network in the TCA-BP cocrystal containing m-xylene, 4.

A different set of inputs, for example IT = [.5,. 1,. 1], would have yielded an output of 0 (IT is the transpose of vector I). The same principle of information feed-forward, weighted sums and transformation applies with multiple units in each layer and, indeed, with multiple layers. Multiple layered networks will be discussed in the next chapter. 

The field of artificial neural networks is a new and rapidly growing field and, as such, is susceptible to problems with naming conventions. In this book, a perceptron is defined as a two-layer network of simple artificial neurons of the type described in Chapter 2. The term perceptron is sometimes used in the literature to refer to the artificial neurons themselves. Perceptrons have been around for decades (McCulloch Pitts, 1943) and were the basis of much theoretical and practical work, especially in the 1960s. Rosenblatt coined the term perceptron (Rosenblatt, 1958). Unfortunately little work was done with perceptrons for quite some time after it was realized that they could be used for only a restricted range of linearly separable problems (Minsky Papert, 1969). 

Granjeon and Tarroux (1995) studied the compositional constraints in introns and exons by using a three-layer network, a binary sequence representation, and three output units to train for intron, exon, and counter-example separately. They found that an efficient learning required a hidden layer, and demonstrated that neural network can detect introns if the counter-examples are preferentially random sequences, and can detect exons if the counter-examples are defined using the probabilities of the second-order Markov chains computed in junk DNA sequences. 

Mezard, M. Nadal, J. P. (1989). Learning in feedforward layered networks The Tilting algorithm. J Phys A 22,2193-203. 

---