Enhancement of Class Prediction by Ensemble Voting Methods

5 ENHANCEMENT OF CLASS PREDICTION BY ENSEMBLE VOTING METHODS 

We introduced several classification methods based on ensemble voting in Section 6.2. In Section 6.6, we will see that these methods tend to perform well compared to methods based on a single classifier. In this section we demonstrate why and how the ensemble approach enhances the class prediction. 

A motivation for ensembles is that a combination of the outputs of many weak classifiers produces a powerful committee (Hastie et al., 2001). We assume independence among the n classifiers, where n is odd. We note that making n odd prevents ties. 

Let Xi denote a random variable indicating a correct classification by the ith classifier. If the prediction accuracy of each classifier is p, then X, BemouUi(p). The number of accurate classifications by ensemble majority voting is Y = Xi binomial(n, p). We letn = 2k+ 1, where fe is a normegative integer and define A = P(Y k + 1). Then the prediction accuracy of the ensemble classification by a majority voting is 

If the classifiers in the ensemble are correlated, then we can use the beta-binomial model (Williams, 1975). This model allows only positive correlafion p in order to satisfy Var(p) 0. Prentice (1988) showed that the beta-binomial model 

---