Random forest algorithm

Woolfitt, A. Moura, H. Barr, J. De, B. Popovic,T. Satten, G. Jarman, K. H. Wahl, K. L., Differentiation of Bacillus spp. by MALDI-TOF mass spectrometry using a bacterial fingerprinting algorithm and a random forest classification algorithm. Presented at 5th ISIAM Meeting, Richland, WA 2004. 

In this study, a machine learning model system was developed to classify cell line chemosensitivity exclusively based on proteomic profiling. Using reverse-phase protein lysate microarrays, protein expression levels were measured by 52 antibodies in a panel of 60 human cancer cell (NCI-60) lines. The model system combined several well-known algorithms, including Random forests, Relief, and the nearest neighbor methods, to construct the protein expression-based chemosensitivity classifiers. 

Bagging alone utilizes the same full set of predictors to determine each split. However the RF algorithm applies another judicious injection of randomness namely by selecting a random subset of the predictors for each split (Breiman 2001). The number of predictors to try at each split is known as m y. While this becomes new parameter, typically (classification) or (regression) works quite well. RFs are not overly sensitive to rr. lATiile Bagging is a special case of random forest where m, = k. 

For Kappa statistics, the RF algorithm has a value of 0.77, which is higher than all other algorithms. This indicates that the raters have greater chance of agreement than other algorithms. For the Mean Absolute Error (MAE), random forest is also better than other algorithms except for the... 

Table 2. Comparison of Random Forest with Other Algorithms.

They mainly consist of a training data set and analyse this training data to learn relationships between data elements to produce an inferred function. They involve algorithms such as Bayesian statistics, decision tree (DT) learning, support vector machine (SVM), random forest (RF) and nearest neighbour algorithms. 

The second approach leaves the data fixed, and introduces a random element into the selection of the splitting variable. So if, for example, predictors x29, x47, and x82 were the only significant splitters and had multiplicity-adjusted p values of say 2 x 10-7, 5 x 10-8, and 3 x 10-3, the conventional greedy algorithm would pick x47 as the splitting variable as it was the most significant. The RRP procedure would pick one of these three at random. Repeating the analysis with fresh random choices would then lead to a forest of trees different random choices will create different trees. 

The Bivariate Marginal Distribution Algorithm (BMDA) [14] is an example of a second order model. It does not build a fully connected model like the HEDA, but discovers a sparse set of second order dependencies between variable pairs. The sparse nature of the connections means that subsets of variables may be completely separate from the rest, meaning that a set of graphs (known as a forest) is produced. The more sparse the network, the lower its capacity for storing attractors. The connections in the BMDA are conditional probabilities, making the model more akin to a Bayesian network, whereas the HEDA structure is more like that of a Markov Random Field. 

---