Neural Network Fundamentals and Methods

Initialize all node connection weights Wjj to some small random values. 

Input a training example Y and corresponding output values Vj, where m is the layer number, i is the node number, and T represents the target or desired output state. 

Propagate the initial signal forward through the network using  

This fimction is continuous and varies monotonically from a lower bound of 0 to an upper bound of 1 and has a continuous derivative. The transfer function in the output layer can be different from than that used in the rest of the network. Often, it is linear, f(NET)=NET, since this speeds up the training process. On the other hand, a sigmoid function has a high level of noise immunity, a feature that can be very useful. Currently, the majority of current CNNs use a nonlinear transfer function such as a sigmoid since it provides a number of advantages. In theory, however, any nonpolynomial function that is bounded and differentiable (at least piecewise) can be used as a transfer 

The feedforward propagation (Eq. 2) is continued for each i and m until the final outputs V° have all been calculated. 

---