CoMFA errors

Ten years after the original CoMFA report [4], more than 100 CoMFA models have been reported on enzyme binding affinity and more than 100 on receptor binding affinity, respectively [31]. An alphabetical list of the 383 CoMFA papers published between 1993 and 1997 is available [83]. Over 200 papers illustrate the use of CoMFA and other 3D-QSAR methods in estimating AG d, with an average prediction error of 0.6-0.7 log units (0.85-1 kcal) for external sets of compounds [31]. This estimate, however, is unlikely to reflect the predictive power of CoMFA models for novel classes of compounds. Care needs to be exercised that the prediction step uses interpolation,... 

CoMFA results are difficult to compare with each other because of the different fields, box sizes, and other options. In addition to this, prior CoMFA versions (up to 5.4) contained an error in the calculation of the electrostatic fields [1012]. Autoscaling of variables should be avoided PLS analysis may produce wrong results if individual grid points largely reduce their variance in the cross-validation, which seemingly occurs quite often [1013] in the CoMFA cross-validation wrong... 

Ungwitayatorn et al. [240] reported the 3D-QSAR CoMFA/CoMSIA studies for a series of 30 Chromone derivatives of HIVPI. The dataset was divided into a training/test set of 30/5 compounds based on the distribution of biological activity and the variety of substitution pattern. Superposition and field fit alignment criteria were used for model development in CoMFA. The best predictive CoMFA model with steric (46%) and electrostatic (54%) fields gave cross-validated of 0.763 and non-cross-validated of 0.967 with a standard error of estimate (S) of 5.092. The PLS protocol and stepwise procedure were... 

The number of significant PLS components is established by testing the significance of each additional dimension (PLS component). This is done to avoid overfitted QSARs, which may exhibit lesser, or no, validity. The optimal number of PLS components to be used in conventional analyses is typically chosen from the analysis with the highest cross-validated value, and for component models with identical values, the model having the smallest standard error of prediction, PRESS (see also the following section). Unlike spectroscopic data, where a PLS model typically has more than 10 components, models in 3D-QSAR tend to exhibit less complexity. As a rule of thumb, two to four components should suffice when CoMFA standard fields are used." ... 

Recommendations for CoMFA studies and 3D QSAR publications have been defined. These recommendations should help to avoid the most common errors and pitfalls and should ease the reproduction of CoMFA results by other scientists in a short version they are summarized below. 

The results of CoMFA and CoMSIA analysis are summarized in Table 7.1. The CoMFA PLS analysis yielded a high cross-validated correlation coeflftcient (f of 0.872 with standard error of prediction SEP of 0.383. The non-cross-validated PLS analysis gave a conventional of 0.974 with SE of 0.172. These values indicated a good statistical correlation and reasonable predictability of the CoMFA model. 

Eleven compounds that were not included in the training set were selected as a test data set to validate the QSAR models. All of the test compounds were well predicted. The mean and standard deviation of prediction errors were 0.28 and 0.005 for the CoMFA model, and only 0.33 and 0.011 for the CoMSIA model. The predictive which was analogous to the cross-validated correlation coefficient q, was 0.883 for the CoMFA and 0.908 for the CoMSIA, suggesting a high reliability of these models. 

---