Backpropagation algorithm

This is useful for pattern recognition. The parameter / influences the nonlinearity of the hidden layer transfer function. The training of the NN is then based on the feedforward and backpropagation algorithm, in which the weighing factors Wn are adjusted during the NN learning in order to minimize the difference between the desired output D and the actual output Y. 

M-CASE/BAIA (see text). BP-ANN = three-layer feedforward artificial neural network trained by the backpropagation algorithm, PAAN = probabilistic artificial neural network, CPANN = counterpropagation artificial neural network. 

J Leonard and MA Kramer. Improvement of the backpropagation algorithm for training neural networks. Comput. Chem. Engg., 14(3) 337-341, 1990. 

The backpropagation algorithm [27] is applied to determine the gradient This... 

Multilayer perceptrons are used nowadays in connection with the backpropagation algorithm. In analytics, more than 90% of applications are based on this learning algorithm. A first approach for this forward directed learning algorithm was made by Werbos (1974). This was further developed by McClelland and Rumelhart [11] (Figure 8.12). 

The optimum number of neurons in the hidden layer varied with the type of training algorithm. If a Bayesian regulation backpropagation algorithm is used, the optimum number of... 

Figure 8.16 Decision boundaries of a feedforward neural network trained by a Bayesian regulation (a) and a conjugate gradient backpropagation algorithm (b).

J.-H. Wang, J.-H. Jiang, and R.-Q. Yu, Chem. Intel Lah. Syst., 34, 109 (1996). Robust Backpropagation Algorithm as a Chemometric Tool to Prevent Overfitting to Outliers. 

The backpropagation algorithm has a tendency for oscillation. To smooth the process, the weights increment Aw y can be modified according to Rumelhart, Hinton, and Wiliams (1986)... 

Note that for small errors, Eq. (19.31) converges to the derivative of activation function at the point of the output value. With an increase of system dimensionality, the chances for local minima decrease. It is believed that the described phenomenon, rather than a trapping in local minima, is responsible for convergency problems in the error backpropagation algorithm. 

The second step of training is the error backpropagation algorithm carried on only for the output layer. Since this is a supervised algorithm for one layer only, the training is very rapid, 100-1000 times faster than in the backpropagation multilayer network. This makes the radial basis-function network very attractive. Also, this network can be easily modeled using computers, however, its hardware implementation would be difficult. 

A schematic diagram of the neural network-based adaptive control technique is shown in Fig. 4.9. A neural network identification model is trained using a static backpropagation algorithm to generate p(fc + 1), given past values of y and u. The identification error is then used to update the weights of the neural identification model. The control error is used to update the... 

Different feedforward neural networks with three layers have been tested to describe the particulate formation in the KPP furnace. A linear activation function has been used in the first layer and tan-sigmoid activation functions were used in the hidden and output layers. Training has been accomplished through 1000 epochs using a backpropagation algorithm. 

---