Transfer function in the hidden units

The transfer function of the hidden units in MLF networks is always a sigmoid or related function. As can be seen in Fig. 44.5b, 0, represents the offset, and has the same function as in the simple perceptron-like networks. P determines the slope of the transfer function. It is often omitted in the transfer function since it can implicitly be adjusted by the weights. The main function of the transfer function is modelling the non-linearities in the data. In Fig. 44.11 it can be seen that there are five different response regions in the sigmoidal function  

B NETy C the response varies almost linearly with NET, 

The region from A to D is called the dynamic range. The regions 2 and 4 constitute the most imfwrtant difference with the hard delimiter transfer function in perceptron networks. These regions rather than the near-linear region 3 are most important since they assure the non-linear response properties of the network. It may 

When the MLF is used for classification its non-linear properties are also important. In Fig. 44.12c the contour map of the output of a neural network with two hidden units is shown. It shows clearly that non-linear boundaries are obtained. Totally different boundaries are obtained by varying the weights, as shown in Fig. 44.12d. For modelling as well as for classification tasks, the appropriate number of transfer functions (i.e. the number of hidden units) thus depends essentially on the complexity of the relationship to be modelled and must be determined empirically for each problem. Other functions, such as the tangens hyperbolicus function (Fig. 44.13a) are also sometimes used. In Ref. [19] the authors came to the conclusion that in most cases a sigmoidal function describes non-linearities sufficiently well. Only in the presence of periodicities in the data 

---