Can We Tell when Good Predictions Are Made

Whatever the results of training sets, and test sets, the users ask what is the best model to use The question is difficult, and even impossible, to answer from the published data on training and validation sets. This is because the performance of any model on any particular test set will heavily depend on the distance of the test set compounds to the training set model space, as well as the inherent predictive poser of the model itself, as has been described in the previous section. The only way to answer this question is for the users to try the models on their own data sets. 

So why cannot we improve the models Johnson [27] attempts to answer this disturbing question. Although his discussion is centered on QSAR, it has many common points relevant to QSPR, which is used in the field of solubility modeling. 

He concludes that we should not be led by mere statistics that too often dictates what is good or wrong, but rather by scientific rationale, experimental design, and personal observation. 

The literature and commercial companies abound with computational solubility models. Many data sets have been studied, with many different descriptor sets, and using a multitude of statistical methods. It appears that diverse drug-like data sets are often predicted by our best methods with an RMSE of 0.8-1 log unit. This compares with an error in replicate measurements of approximately 0.5 log unit. A common view is that there is still room for improvement in the computational modeling of solubility. There are a number of suggestions that the quality control of the ideal data set is still lacking. This may be true for some literature data set compilations, but it is 

3 Dressman, J.B., Vertzoni, M., Goumas, K. and Reppas, C. (2007) Estimating drug solubility in the gastrointestinal tract. Advanced Drug Delivery Reviews, 59 (7), 

---