Underfitting and Overfitting Problems of Machine Learning

Since too narrow scope of function set used cannot give good result, it is natural to think that the use of wider scope of function set in data processing may give better results of machine learning, and that an all-inclusive function set can be used to avoid underfitting problem. 

Here we will demonstrate the problem of overfitting with some examples of chemical data processing. Table 1.2 demonstrates a set of data about the preparation of bismuth-based high-temperature superconductors. And Table 1.3 demonstrates a set of data about the preparation of VPTC ceramic semiconductors. In these tables, the samples of class 1 are those with good properties, and those of class 2 with unsatisfactory properties. 

Sample No. Class Tlh03% ExcessTi02% Sintering time (hr) Relative cooling rate 

The result of data processing for the data set in Table 1.3 is quite similar. Although the rate of correctness in training process increases very quickly (it means that the structure of the data set is relatively simple and can be imitated by using ANN very easily), the minimum number of errors in prediction test (by LOO cross-validation method) of ANN is still higher than that of support vector machine, as shown in Fig. 1.4. 

Therefore, in machine learning work, we have two enemies underfitting and overfitting. The enlargement of the scope of hypothesis functions can only avoid the underfitting problem. However, it often makes overfitting becoming more serious problem. 

---