Nets Backpropagation

The learning problem is to find the weights vectors tv such that the vector of the computed outputs of all units, o, is as close as possible, if not equal, to the vector of the desired output of all units, y, for all the available input vectors x. The system works in a manner similar to the simple perceptron 

The generalized delta rule for the feth output unit is given by 

Similarly, the same rule for the /th hidden unit is given by 

Note that the 8 values on an internal node may be computed based on the 8 values of the superior nodes, which is the theoretical reason for evaluating in the first place the errors of the output layer, and later propagating these errors 

Let the hidden units evaluate their output using the pattern. 

---

The previous section introduced the backpropagation rule for multi-layer percep-trons. This section briefly discusses tfie model development cycle necessary ftu-obtaining a properly functioning net. It also touches upon some of the available heuristics for determining the proper size of hidden layers. 

Fundamentally, all feed-forward backpropagating nets follow the same five basic steps of a model development cycle ... 

The obvious lesson to be taken away from this amusing example is that how well a net learns the desired associations depends almost entirely on how well the database of facts is defined. Just as Monte Carlo simulations in statistical mechanics may fall short of intended results if they are forced to rely upon poorly coded random number generators, so do backpropagating nets typically fail to ac hieve expected re.sults if the facts they are trained on are statistically corrupt. 

Hernandez, E., and Arkun, Y., A study of the control relevant properties of backpropagation neural net models of nonlinear dynamical systems. Comput. Chem. Eng. 16, 227 (1992). 

For kriging models, the structure is defined by the set of independent variables selected - including quadratic terms - and the selection of the correlation model. The parameter estimation is performed by a maximum likelihood procedure. For neural nets, the activation function to be used is defined a priori. The structure is completed by the selection of the number of neurons in the hidden layer. A backpropagation procedure has been used for training. 

Nestor s NDS is a high-end development system for PC/AT s and for Apollo and Sun workstations. Unlike the other neural net shells, NDS is based on a propietary model on which Nestor holds a patent. Nestor claims that networks based on its model can be trained significantly faster than models based on backpropagation. 

The backpropagation net is an example of a supervised learning neural network. Its applications in analytics are given in Table 8.4. 

Fig. 6 Predicted vs. adjusted (real) concentration of strychnine in embryonic mice spinal cord cultures, evaluated with a backpropagation artificial neural network [22, 23, 25]. (a) The net was trained with 2/3 of the data...

Given a function to implement as a neural network, there are a number of (currently) ill-understood and unconventional stages in the process. Instead of designing an algorithm to meet the specification, we train a network to do so, and in both cases we test the implementation to determine whether it is acceptable. Currently, we are using Multilayer Perceptron nets trained with the backpropagation algorithm [8]. 

---