Feed-forward network architectures

Current feed-forward network architectures work better than the current feed-back architectures for a number of reasons. First, the capacity of feed-back networks is unimpressive. Secondly, in the running mode, feed-forward models are faster, since they need to make one pass through the system to find a solution. In contrast, feed-back networks must cycle repetitively until... 

Figure 1.16. Schematic architecture of a three layer feed-forward network...

Figure 13.9 Architecture of a fully connected feed-forward network. Formal neurons are drawn as circles, and weights are represented by lines connecting the neuron layers. Fan-out neurons are drawn in white, sigmoidal neurons in black.

Neural network architectures 2L/FF = two-layer, feed forward network (i.e., perceptron) 3L or 4L/FF = three- or four-layer, feed-forward network (i.e., multi-layer perceptron). 

An important factor in the popularity of feed-forward networks is that it has been shown that a continuous valued neural network with a continuous differentiable non-linear transfer function can approximate any continuous function arbitrarily well (Cybenko, 1989). The feedforward architecture shown in Fig. 27.1 is typically used for steady-state functional approximation or one-step-ahead dynamic prediction. However, if the model is to be used to predict also more than one time step ahead, recurrent nemal networks should be used, in which delayed outputs are used as neuron inputs... 

Neural networks can be broadly classified based on their network architecture as feed-forward and feed-back networks, as shown in Fig. 3. In brief, if a neuron s output is never dependent on the output of the subsequent neurons, the network is said to be feed forward. Input signals go only one way, and the outputs are dependent on only the signals coming in from other neurons. Thus, there are no loops in the system. When dealing with the various types of ANNs, two primary aspects, namely, the architecture and the types of computations to be per-... 

As a chemometric quantitative modeling technique, ANN stands far apart from all of the regression methods mentioned previously, for several reasons. First of all, the model structure cannot be easily shown using a simple mathematical expression, but rather requires a map of the network architecture. A simplified example of a feed-forward neural network architecture is shown in Figure 8.17. Such a network structure basically consists of three layers, each of which represent a set of data values and possibly data processing instructions. The input layer contains the inputs to the model (11-14). 

It should be mentioned that there are other types of network architectures that can be used, but the feed-forward structure is shown here due to its relative simplicity and high relevance to quantitative method building. 

Figure 12.1. Multi-layered Feed Forward Neural Network Architecture...

There are literally dozens of kinds of neural network architectures in use. A simple taxonomy divides them into two types based on learning algorithms (supervised, unsupervised) and into subtypes based upon whether they are feed-forward or feedback type networks. In this chapter, two other commonly used architectures, radial basis functions and Kohonen self-organizing architectures, will be discussed. Additionally, variants of multilayer perceptrons that have enhanced statistical properties will be presented. 

The way in which the neurons are interconnected is referred to as the network architecture or topology. A variety of network architectures has been developed for different applications, but one of the most common is the so-called multilayer perceptron (MLP) network, illustrated in Fig. 2. This is a feedforward network, the feed forward meaning that information is passed in one direction through the network from the inputs, through various hidden layers, to the outputs. The inputs are simply the input variables and, as stated above, in the case of formulation these correspond to ingredients, ingredient amounts, and processing conditions. The hidden layers are made up of perceptrons. Typically, one hidden layer is adequate to learn the relationships in most data sets two hidden layers should enable all... 

FIGURE 10.2 TargetP localization predictor architecture. TargetP is built from two layers of feed-forward neural networks and, on top, a decision-making unit, taking into account cutoff restrictions (if opted for) and outputting a prediction and a reliability class, RC, which is an indication of prediction certainty (see the text). The nonplant version lacks the cTP network unit in the first layer and does not have cTP as a prediction possibility. 

Larger architectures emerged since then, among them, the feed-forward multilayer perceptron (MLP) network has became the most popular network architecture (Hertz et al. 1991). The disposition of neurons in such ANN is quite different from the disposition in the brain they are disposed in layers with different number of neurons each. Layers are named according to their position in the architecture an MLP network has an input layer, an output layer and one or more hidden layers between them. Interconnection between neurons is accomplished by weighted connections that represent the synaptic efficacy of a biological neuron. 

The last type of ANN classifier employed is the probabilistic (P) network, which is a feed-forward neural network in which a Bayesian decision strategy for classifying input vectors is implemented (Freeman, 1994). The P network has a 16-element input neuron and a 4-element output neuron. Figure 3.6 depicts the architecture of a P network. 

Based on computation done, it is observed that neural network architecture with feed forward backpropagation method of 3 layers consists of one input layer, one hidden... 

Figure 9.7. Architecture of a three-layer forward-feed neural network. The neurons in the input layer simply pass their inputs to the next layer without addition or transfer function. The conventions are the same as those used in Figure 9.6. (Reproduced from [26], by permission of John Wiley Sons, Ltd. copyright 2002.)...

Figure 2 Architecture of a multilayer feed-forward neural network...

Figure 3 The architecture of the feed-forward neural network used in multispectrum interpretation. The input layer is a vector representing the IR and C NMR spectral data and the molecular formula. Each of the output units is assigned to one structural feature...

The architecture of the static three layers of feed-forward neural network is shown in Fig. 1. This model is trained with the process dynamic parameto s and measured quality at the same time interval. Therefore, it works as a static model. The output of SFNN is calculated according to... 

---