Tree-based classification

We previously applied Random Forests (RF), a tree-based classification and regression method to pathway analysis of gene expression data.46 The proposed methods allow researchers to rank the significance of biological pathways as well as discover important genes in the same process. 

Another nonparametric regression method is CART (classification and regression trees). The basic concepts were outlined in Discriminant Analysis Section about tree-based classification. We remember from that chapter that CART is a recursive binary partition method based on a simple model constant for each region. If the residual sums of squares of responses is minimized. 

Resubstitution of the benzanthracene concentrations reveals perfect fit, that is, the calibration mean squared error (Eq. (6.106)) is almost zero. Estimation of predictions from a single tree by leave-one-out cross-validation reveals a mean squared prediction error of 0.428. A plot of the recovery function for the individual predictions is given in Eigure 6.16a. Further improvement of the model is feasible if again ensemble methods (cf Tree-Based Classification Section) are applied. Figure 6.16b shows the recovery function for a bagged model with a smaller prediction error of 0.337. 

Bootstrapping is restrained to the use of the original data set to estimate confidence intervals. Part of the data (rows of the data matrix) is sorted out and used for later predictions, The missing rows in the matrix are replaced randomly by data vectors kept. The latter vectors are then used twice in a computational run (cf. Tree-Based Classification Section). 

The validity of the above classification is emphasized by the fact that the parsimonious tree based on the full length GH1 (Fig. 4b) has the same topology as the tree based on comparison of the size of the insert in the wing subdomain [77]. Both trees separate HI into two main branches the branch of Dictiostelium and plants and the branch of fungi and animals. 

The total number of trees is n which is the sum of n forward trees, n reverse trees and n(n — 2) mixed trees. This classification of the spanning tree gives a simple algorithm for calculating the base determinant for a one route mechanism. 

Shedden KA, Taylor JM, Giordano TJ, et al. Accurate molecular classification of human cancers based on gene expression using a simple classifier with a pathological tree-based framework. Am J Pathol. 2003 163 1985-1995. 

The models of tree-based methods can be improved by ensemble methods, where several different decision trees are aggregated to an ensemble and the shghtly differing classification results are averaged. Currently, the most popular ensemble methods are bagging and boosting. 

However, the instability of tree-based methods implies also here a much higher error in case of cross-validation by the simple leave-one-out method, that is, the cross-validated fraction of misclassified objects for CART is with 2.25%, 10 times higher than the resubstitution error. This error can be only reduced if ensemble methods are included in the model budding step. A bagged CART model revealed a cross-validation error of only 1.0% (Figure 5.38e). The fraction of misclassifica-tions for the cross-validated models increases for QDA, SVM, and A-NN to 5.5%, 5.0%, and 4.75%, respectively. The cross-validated classifications by LDA reveal 58.8% of misclassified objects as expected from the type of data. 

FIGURE 7.3 Phylogenetic tree of lower vertebrates, from Haeckel (l866 Plate 7 [part]).This is essentially a phylogenetic tree based on the ideas of pre-Darwinian classification. Notice that there are no animals of equal rank placed on the branches. There are no specified ancestors. Ideas of ancestry are contained solely in the graphic depiction of branches. 

Genera in bold = significant for this book (studies on secondary metabolites published) genera in standard type = no phytochemical results available underlined genera = classification different from the phylogenetic tree based on (DNA) molecular analysis (see Fig. 2.2)... 

Purdy [91] used the technique to predict the carcinogenicity of organic chemicals in rodents, although his model was based on physicochemical and molecular orbital-based descriptors as well as on substructural features and it used only a relatively small number of compounds. His decision tree, which was manual rather than computer based, was trained on 306 compounds and tested on 301 different compounds it achieved 96% correct classification for the training set and 90% correct classification for the test set. 

A classification decision tree allows one to predict in a sequential way the y value (or corresponding conditional probabilities) that is associated with a particular x vector of values. At the top node of the tree (A in Fig. 3), a first test is performed, based on the value assumed by a particular decision variable (jCj). Depending on the outcome of this test, vector X is sent to one of the branches emanating from node A. A second test follows, being carried out at another node (B), and over the values of the same or a different decision variable (e.g., JCg). This procedure is... 

The preceding strategy for the construction of decision trees provides an efficient way for inducing compact classification decision trees from a set of (x, y) pairs (Moret, 1982 Utgoff, 1988 Goodman and Smyth, 1990). Furthermore, tests based on the values of irrelevant variables are not likely to be present in the final decision tree. Thus, the problem dimensionality is automatically reduced to a subset of decision variables that convey critical information and influence decisively the system performance. 

---