Neural input layer

Figure 6 Schematic of a typical neural network training process. I-input layer H-hidden layer 0-output layer B-bias neuron.

Artificial neural networks (ANN) are computing tools made up of simple, interconnected processing elements called neurons. The neurons are arranged in layers. The feed-forward network consists of an input layer, one or more hidden layers, and an output layer. ANNs are known to be well suited for assimilating knowledge about complex processes if they are properly subjected to input-output patterns about the process. 

Fig. 7. Artificial neural network model. Bioactivities and descriptor values are the input and a final model is the output. Numerical values enter through the input layer, pass through the neurons, and are transformed into output values the connections (arrows) are the numerical weights. As the model is trained on the Training Set, the system-dependent variables of the neurons and the weights are determined.

Figure 5.3 Internal organisation of an artificial neural network. In general, there is a neuron per original variable in the input layer of neurons and all neurons are interconnected. The number of neurons in the output layer depends on the particular application (see text for details).

As a chemometric quantitative modeling technique, ANN stands far apart from all of the regression methods mentioned previously, for several reasons. First of all, the model structure cannot be easily shown using a simple mathematical expression, but rather requires a map of the network architecture. A simplified example of a feed-forward neural network architecture is shown in Figure 8.17. Such a network structure basically consists of three layers, each of which represent a set of data values and possibly data processing instructions. The input layer contains the inputs to the model (11-14). 

Figure 8.2 A motor neuron (a) and small artificial neural network (b). A neuron collects signals from other neurons via its dendrites. If the neuron is sufficiently activated, it sends a signal to other neurons via its axon. Artificial neural network are often grouped into layers. Data is entered through the input layer. It is processed by the neurons of the hidden layer and then fed to the neurons of the output layer. (Illustration of motor neuron from Life ART Collection Images 1989-2001 by Lippincott Williams Wilkins used by permission from SmartDraw.com.)...

Artificial neural networks often have a layered structure as shown in Figure 8.2 (b). The first layer is the input layer. The second layer is the hidden layer. The third layer is the output layer. Learning algorithms such as back-propagation that are described in many textbooks on neural networks (Kosko 1992 Rumelhart and McClelland 1986 Zell 1994) may be used to train such networks to compute a desired output for a given input. The networks are trained by adjusting the weights as well as the thresholds. 

Figure 8.4 Funt et al. (1996) transform the colors of the input image to rg-chromaticity space. The input layer of the neural network samples the triangular shaped region of the chromaticity space. The network consists of an input layer, a hidden layer, and an output layer. The two output neurons estimate the chromaticity of the illuminant.

Figure 8.14 Neural architecture of Usui and Nakauchi (1997). The architecture consists of four layers denoted by A, B, C, and D. The input image is fed into the architecture at the input layer. Layer D is the output layer. (Redrawn from Figure 8.4.1 (page 477) Usui S and Nakauchi S 1997 A neurocomputational model for colour constancy. In (eds. Dickinson C, Murray I and Carded D), John Dalton s Colour Vision Legacy. Selected Proceedings of the International Conference, Taylor Francis, London, pp. 475-482, by permission from Taylor Francis Books, UK.)...

Fig. 2. Structure of an artificial neural network. The network consists of three layers the input layer, the hidden layer, and the output layer. The input nodes take the values of the normalized QSAR descriptors. Each node in the hidden layer takes the weighted sum of the input nodes (represented as lines) and transforms the sum into an output value. The output node takes the weighted sum of these hidden node values and transforms the sum into an output value between 0 and 1.

Neural networks are essentially non-linear regression models based on a binary threshold unit (McCulloch and Pitts, 1943). The structure of neural networks, called a perception, consists of a set of nodes at different layers where the node of a layer is linked with all the nodes of the next layer (Rosenblatt, 1962). The role of the input layer is to feed input patterns to intermediate layers (also called hidden layers) of units that are followed by an output result layer where the result of computation is read-off. Each one of these units is a neuron that computes a weighted sum of its inputs from other neurons at a previous layer, and outputs a one or a zero according to whether the sum is above or below a... 

Using 51-nucleotide sequence windows, Nair et al. (1994) devised a neural network to predict the prokaryotic transcription terminator that has no well-defined consensus patterns. In addition to the BIN4 representation (51 x 4 input units), an EIIP coding strategy was used to reflect the physical property (Le., electron-ion interaction potential values) of the nucleotide base (51 units). The latter coding strategy reduced the input layer size and training time but provided similar prediction accuracy. 

The basic information of protein tertiary structural class can help improve the accuracy of secondary structure prediction (Kneller et al., 1990). Chandonia and Karplus (1995) showed that information obtained from a secondary structure prediction algorithm can be used to improve the accuracy for structural class prediction. The input layer had 26 units coded for the amino acid composition of the protein (20 units), the sequence length (1 unit), and characteristics of the protein (5 units) predicted by a separate secondary structure neural network. The secondary structure characteristics include the predicted percent helix and sheet, the percentage of strong helix and sheet predictions, and the predicted number of alterations between helix and sheet. The output layer had four units, one for each of the tertiary super classes (all-a, all-p, a/p, and other). The inclusion of the single-sequence secondary structure predictions improved the class prediction for non-homologous proteins significantly by more than 11%, from a predictive accuracy of 62.3% to 73.9%. 

A network is composed of units or simple named nodes, which represent the neuron bodies. These units are interconnected by links that act like the axons and dendrites of their biological counterparts. A particular type of interconnected neural net is shown in Fig. 5.12. In this case, it has one input layer of three units (leftmost circles), a central or hidden layer (five circles) and one output (exit) layer (rightmost) unit. This structure is designed for each particular application, so the number of the artificial neurons in each layer and the number of the central layers is not a priori fixed. 

Figure 18 A neural network, comprising an input layer (I), a hidden layer (H), and an output layer (O). This is capable of correctly classifying the analytical data from Table 1. The required weighting coefficients are shown on each connection and the bias values for a sigmoidal threshold function are shown above each neuron...

The network has three hidden layers, including a bottleneck layer which is of a smaller dimension than either the input layer or the output layer. The network is trained to perform an identity mapping by approximating the input information at the output layer. Since there are fewer nodes in the bottleneck layer than the input or output layers, the bottleneck nodes implement data compression and encode the essential information in the inputs for its reconstruction in subsequent layers. In the NLPCA framework and terminology, autoassociative neural networks seek to provide a mapping of the form... 

The architecture of an ANFIS model is shown in Figure 14.4. As can be seen, the proposed neuro-fuzzy model in ANFIS is a multilayer neural network-based fuzzy system, which has a total of five layers. The input (layer 1) and output (layer 5) nodes represent the descriptors and the response, respectively. Layer 2 is the fuzzification layer in which each node represents a membership. In the hidden layers, there are nodes functioning as membership functions (MFs) and rules. This eliminates the disadvantage of a normal NN, which is difficult for an observer to understand or to modify. The detailed description of ANFIS architecture is given elsewhere (31). 

---