Random forest boosting

Mixed-integer programming hyperboxes classification, Bayes Network, Naive Bayes, Liblinear, LibSVM, RBF network, SMO, Logistic, IBk, Bagging, Ensemble selection, Logit Boost, LMT, NBTree, Random Forest, DTNB... 

GBM, gradient boosting models MLR, multivariate linear regression NN, artificial neural net RF, random forest, n, descriptors, number of descriptors used n, train, size of training set, n, test, size of test set. Fivefold cross-validated results. 

Random Forest methods (Breiman 2001 Random Forests 2001) construct ensembles of trees based on multiple random selections of subsets of descriptors and bootstrapping of compounds. The compounds not selected in a particular bootstrapping are considered as a so-called out of bag set, and used as the test set. The trees are not pruned. Best trees in the forest are chosen for consensus prediction of external compounds. The method can include bagging (Berk 2008 Breiman 1996) and boosting (Berk 2008 Breiman 1998) approaches. 

At each internal tree node, a decision forest randomly can select F feature attributes, and evaluate just those attributes to choose the partitioning attribute. They tend to produce trees larger than trees where all attributes are considered for selection at each node, but different classes will be eventually assigned to different leaf nodes. At each internal tree node, RFs evaluate the quality of all possible partitioning attributes, but randomly select one of the Fbest attributes to label that node based on information gain, etc. RFs are an effective tool in prediction compared with boosting and adaptive bagging. 

---