Overtraining

Another method of detection of overfitting/overtraining is cross-validation. Here, test sets are compiled at run-time, i.e., some predefined number, n, of the compounds is removed, the rest are used to build a model, and the objects that have been removed serve as a test set. Usually, the procedure is repeated several times. The number of iterations, m, is also predefined. The most popular values set for n and m are, respectively, 1 and N, where N is the number of the objects in the primary dataset. This is called one-leave-out cross-validation. 

In all modeling techniques, and neural networks in particular, care must be taken not to overtrain or overfit the model. 

A typical performance behaviour is shown in Fig. 44.16b. The increase of the NSE for the monitoring set is a phenomenon that is called overtraining. This phenomenon can be compared to fitting a curve with a polynomial of a too high order or with a PCR or PLS model with too many latent variables. It is caused by the fact that after a certain number of iterations, the noise present in the training set is modelled by the network. The network acts then as a memory, able to recall... 

Fig. 44.16. (a) Ideal performance curve of a MLF network. The solid line represents the training set the dashed line represents the monitoring set. (b) overtraining of the network begins at point A. (c) Paralysed network, (d) Performance curve of an MLF network for which the training and control samples represent different models. 

From the previous sections it is clear that it is important to use a network with a suitable number of hidden units. When too few hidden units are used, the relationship cannot be modelled properly and the network shows poor performance. Too large a number of hidden units causes severe overtraining. The suitable number of hidden units depends on the problem complexity and on the number of training examples that are available. It must be determined empirically. There are basically three approaches for this ... 

Train and test a network with a certain number of hidden units, based on an educated guess. When the NSE of the network is acceptable and no severe overtraining occurs, the network is suitable. 

It has been known in sports medicine that too much training can bring on a condition known as overtraining or underperformance syndrome (UPS). Symptoms include sleep disturbance, increased perception of effort, decreased effectiveness of the immune system and unexpected poor performance in training or in competition. At present, there is no satisfactory biochemical or physiological explanation for the syndrome. Identification of a biochemical marker... 

If I assert that my memory, however impaired by age, is still quite good, and add that I am a highly overtrained observer, you will quite correctly chide me with hubris because you know that personal recollection is notoriously unreliable. Two people who have seen the same crime or alleged criminals often can t agree on the details well enough to convince a jury of what they saw. 

The physiological similarities between sleep deprivation and the overtraining syndrome (OTS) are not surprising. Simply put, both are due to an imbalance... 

Urhausen A, Kindermann W. Diagnosis of overtraining what tools do we have Sports Med 2002 32 95-102. 

Lehmann M, Foster C, Dickhuth HH, Gastmann U. Autonomic imbalance hypothesis and overtraining syndrome. Med Sci Sports Exerc 1998 30 1140-1145. 

MacKinnon LT. Special feature for the Olympics effects of exercise on the immune system overtraining effects on immunity and performance in athletes. Immunol Cell Biol 2000 78 502-509. 

Hooper SL, Traeger-Mackinnon L, Howard A, Gordon RD, Bachmann AW. Markers for monitoring overtraining and recovery. Med Sci Sports Exerc 1995 27 106-112. 

---