The Standard Back Propagation Algorithm

To explain the back propagation algorithm, a simple feed-forward neural network of three layers (input, hidden and output) is used. The network input will be denoted by Xj, the output of the hidden neurons by Hi, that of the output neurons by i. The weights of the links between the input and the hidden layer are written as w,j, where i refers to the nmnber of the input neuron and j to the number of the hidden neuron. The weights of the links between the hidden and the output layer are denoted as again j stands for the number of the hidden neuron and k for the number of the output neuron. An example network with notation is shown in Fig. 27.3, This network has three input neurons, three hidden neurons and two output neurons, in this case the input layer passes on the inputs, i.e. Ij = Xj. 

The input consists of several patterns that will be labeled by a superscript p, so input i is set to xf when pattern p is presented. An input pattern is one single network input combination, it has one single, however not necessarily unique, network output combirratiorr. This mearts that the training set consists of a certain nitmber of input/output pairs f The nerrrorrs in the network all have the same activation frmctiong(). 

Given pattern p, hidden unit J receives a net input from the input layer as follows  

Therefore output neuron k receives the following input  

In most cases, a linear output function is used, in which case yl is equal to.  

---

After the specific initial structure of the neural network is determined it still needs to be trained to learn the process. Different methods of training exist, with the standard back propagation algorithm (Rttmelhart et al, 1986) being the most popular. This algorithm will be explained in the next paragraph. 

---