Determining the number of hidden units

From the previous sections it is clear that it is important to use a network with a suitable number of hidden units. When too few hidden units are used, the relationship cannot be modelled properly and the network shows poor performance. Too large a number of hidden units causes severe overtraining. The suitable number of hidden units depends on the problem complexity and on the number of training examples that are available. It must be determined empirically. There are basically three approaches for this  

---

Fig. 44.17. Determining the number of hidden units for an MLF network. The whiskers represent the range of the error with different random weight initializations or by cross-validation.

Murata N., Yoshizawa S., Amari S., Network information criterion determining the number of hidden units for an artificial neural network model, IEEE transaction on Neural Networks, 1994,5(6), 865-872. 

---