Bonferroni method

The Bonferroni contrast procedure can also be used for g contrasts. 

The Scheffe method is recommended when the researcher desires to compare all possible contrasts, but the Bonferroni method is used when specific contrasts are desired. 

---

In certain situations, such as antimicrobial time-kill studies, an investigator may be interested in confidence intervals for both bo (initial population) and b (rate of inactivation). In previous examples, confidence intervals were calculated for bo and b separately. Now we discuss how confidence intervals for both bo and bi can be achieved simultaneously. We use the Bonferroni method for this procedure. 

There is a knotty issue in multiple regression with using the Student s t-test for >1 independent predictors. That is, because more than one test was conducted, for example, 0.95 = 0.857 confidence. To adjust for this, the user can undertake a correction process, such as the Bonferroni joint confidence procedure. In our example, there are parameters, if one includes bo. Not all of them need to be tested, but whatever that test number is, we call it as g, where g S6- This is the same formula as the previous one, using the t-table, except that a is divided by 2g, where g is the number of contrasts. 

During April, 2003 the length of the fry from each experiment was measined and one-way ANOVA analysis was conducted on the three sets of data followed hy Tukey tests. Kolmogorov-Smimov tests were conducted on the hatch time data to determine if the distributions differed between the treatments. The P-value was adjusted to 0.03 using the bonferroni method to account for the multiple comparisons being made. Chi-square tests were conducted to determine a hatch time bias for the period of dark. 

The major drawback of this method is that Bonferroni s Inequality is a conservative correction, especially if some of the hypotheses being tested are not independent. When many SNPs in the same gene are evaluated, for example, and are in LD with each other, the Boneferroni correction would not be appropriate, resulting in the possibility of false negatives or failure to detect a true association. A better approach would be to test the true level of significance directly through simulations. 

It tests all linear contrasts among the population means (the other three methods confine themselves to pairwise comparison, except they use a Bonferroni type correlation procedure). 

The technique for obtaining interval estimates for X, discussed in this section, is presented in the paper by Lieberman, Miller, and Hamilton ( 2 ) and based on the Bonferroni inequality ( ) described below Other methods are found in the references ( 23,24 ) ... 

The frequency interpretation of the interval estimates on the unknown amounts is given by the following ( 27 ) With at least 1- a confidence, based on the sampling characteristics of the observations on the standards, at least P proportion of the interval estimates made from a particular calibration will contain the true amounts. The Bonferroni inequality insures the 1-a confidence since the confidence interval about the regression line and the upper bound on cr are each performed using a 1- a/2 confidence coefficient. Hence, the frequency interpretation states that at least (1-a) proportion of the standard calibrations are such that at least P proportion of the intervals produced by the method cover the true unknown amounts. For the remaining a proportion of standard calibrations the proportion of intervals which cover the true unknown values may be less than P. 

Differences in calibration graph results were found in amount and amount interval estimations in the use of three common data sets of the chemical pesticide fenvalerate by the individual methods of three researchers. Differences in the methods included constant variance treatments by weighting or transforming response values. Linear single and multiple curve functions and cubic spline functions were used to fit the data. Amount differences were found between three hand plotted methods and between the hand plotted and three different statistical regression line methods. Significant differences in the calculated amount interval estimates were found with the cubic spline function due to its limited scope of inference. Smaller differences were produced by the use of local versus global variance estimators and a simple Bonferroni adjustment. 

Pocock (1977) developed a procedure which divides the type I error rate of 5 per cent equally across the various analyses. In the example above with two interim looks and a final analysis, Bonferroni would suggest using an adjusted significance level of 0.017 (= 0.05 4- 3). The Pocock method however gives us the correct adjusted significance level as 0.022 and this exactly preserves the overall 5 per cent type I error rate. 

Note that performing multiple hypothesis tests (e.g., Student s t-test) may inflate the false positive rate. That is, the number of genes detected as active by chance alone will increase with the number of genes tested. For example, a microarray with 7000 features would require at least 7000 hypothesis tests per treatment comparison. Several methods have been developed to control the false positive rate, such as the conservative Bonferroni correction and the FDR control method (49). 

Bonferroni s test is the most straightforward of several statistical methodologies that can appropriately be used in the context of multiple comparisons. That is, Bonferroni s test can appropriately be used to compare pairs of means after rejection of the null hypothesis following a significant omnibus F test. Imagine that we have c groups in total. Bonferroni s method makes use of the following inequality ... 

This means that the probability of rejecting at least one of c hypotheses is less than or equal to (thus the term "inequality") the sum of the probabilities of rejecting each hypothesis. This inequality is true even if the events, in this case rejecting one of c null hypotheses, are not independent. Recall from Section 6.2 that, when events are not independent, the probability of intersecting events should be subtracted. Using Bonferroni s method, testing each pair of means with an a level of a = will ensure that the overall type I error rate does not exceed the desired value of a. It follows that the probability of rejecting at least one of c null hypotheses can be expressed as follows ... 

When using Bonferroni s method, the null hypothesis associated with a pairwise comparison is rejected if the calculated test statistic, that is. 

Having introduced Bonferroni s test, we can now return to our earlier example to see how to apply Bonferroni s method to our pairwise comparisons of treatment group means. 

Given that the decisions made from this trial could result in sizeable further investment in the development of the investigational antihypertensive drug, the company would like to minimize its chances of committing a type I error. That is, it would like to maintain an overall type I error of 0.05. As we have just seen in Section 11.7, one analysis that will maintain this desired type I error of 0.05 is Bonferroni s method. 

Bonferroni s method that we have just discussed is perhaps one of the most easily understood methods to maintain an overall type I error, which is one of its advantages. In addition, Bonferroni s method does indeed control the overall type I error rate well, such that it is guaranteed to be < a. However, like many items that we discuss in this book, it has its disadvantages as well as its advantages. 

Bonferroni s test is overly conservative, in that the critical values required for rejection need not be as large as they are. In other words, using a less conservative method may result in more null hypotheses being rejected. The reason that Bonferroni s method is so conservative is that it does not in any way account for the extent of correlation among the various hypotheses being tested. If a method could take into account the overlap, or lack thereof, of the various hypotheses, the critical values would not need to be defined as narrowly as with Bonferroni s. In this section, we therefore discuss another analytical strategy for multiple comparisons, Tukey s honestly significant difference (HSD) test. 

Bonferroni s method for testing pairs of means (maintaining an overall type I error rate of a) involved comparing the absolute differences in means to the MSD, which was defined as a function of ... 

The statistical interpretations of these results are the same as with Bonferroni s method. The 20 and 30 mg doses both resulted in a statistically significantly greater SBP reduction than the 10 mg dose. There was insufficient evidence to claim that there is a statistically significant difference between the 20 mg and the 30 mg doses. 

It should be noted that these are not the only acceptable methods applicable to multiple comparisons from an ANOVA. In each individual case, the choice among possible approaches is largely dependent on the study design. For example, Dunnett s test can be used when the only comparisons of interest are each test treatment versus a control (for example, in a placebo-controlled, dose-ranging study). Like Tukey s test, Dunnett s method is more powerful than Bonferroni s. In general, other methods gain power compared with Bonferroni s method by... 

---