Algorithm back-propagation

As described below, the required behaviour is taught to the neural net by back propagation. This procedure is carried out by exposing the network to sets consisting of one input vector and its corresponding output vector. By an iterated procedure of trial and error, the convergence to determine the weight values that minimizes a prescribed error value is then achieved. 

The adjustment on weights w j is done using the sensitivity of the error with respect to that weight, as  

The expression for the weight change is obtained from Eqs. (5.171) and (5.172) replacing them in the relation (5.174)  

In this case a and y represent, respectively, the rate factor in output and the scaling factor of the net. These relations are also related to the sigmoid threshold function. 

---

The Back-Propagation Algorithm (BPA) is a supervised learning method for training ANNs, and is one of the most common forms of training techniques. It uses a gradient-descent optimization method, also referred to as the delta rule when applied to feedforward networks. A feedforward network that has employed the delta rule for training, is called a Multi-Layer Perceptron (MLP). 

Figure 16 Root-mean-squared error progression plot for Fletcher nonlinear optimization and back-propagation algorithms during training.

Pseudo-Code Implementation The Back-propagation algorithm outlined above may be implemented by following seven steps, to be applied for each pattern p . Assume we have a neural net with L layers (/ 1,2,...,L). Let h represent the output of the neuron in the / layer hf is therefore equal to the input, <7 . The weight of the connection between li- and h[ will be labeled by Wij. 

Leonard, J.. and Kramer, M. A., Improvement of the back propagation algorithm for training neural networks, Comput. Chem. Eng. 14, 337-341 (1990). 

The formula used in implementing the back-propagation algorithm is given by ... 

Na and K) employing different pattern recognition techniques [PCA, linear discriminant analysis (LDA) and ANNs], ANNs were trained by an error back-propagation algorithm and they were found to be very efficient for classifying and discriminating food products. 

In the method using the back-propagation algorithm (see [145] for details), the training pairs are presented successively, and for each of them the weights at the interconnects between the hidden and output layers are adjusted before those between the input and hidden layers. 

Van Ooyen, A. Nienhuis, B. (1992). Improving the convergence of the back-propagation algorithm. NEUNET 5,465-71. 

Chemometric Design Expert software was used to generate and manipulate the factorial data Neural network was carried out using the back propagation algorithm (WEKA) ... 

Neural networks are also being seriously explored for certain classes of optimization applications. These employ parallel solution techniques which are patterned after the way the human brain functions. Statistical routines and back propagation algorithms are used to force closure on a set of cross linked circuits (equations). Weighting functions are applied at each of the intersections. 

---