Bonferroni s test

Bonferroni s test is the most straightforward of several statistical methodologies that can appropriately be used in the context of multiple comparisons. That is, Bonferroni s test can appropriately be used to compare pairs of means after rejection of the null hypothesis following a significant omnibus F test. Imagine that we have c groups in total. Bonferroni s method makes use of the following inequality ... 

Having introduced Bonferroni s test, we can now return to our earlier example to see how to apply Bonferroni s method to our pairwise comparisons of treatment group means. 

Bonferroni s test is overly conservative, in that the critical values required for rejection need not be as large as they are. In other words, using a less conservative method may result in more null hypotheses being rejected. The reason that Bonferroni s method is so conservative is that it does not in any way account for the extent of correlation among the various hypotheses being tested. If a method could take into account the overlap, or lack thereof, of the various hypotheses, the critical values would not need to be defined as narrowly as with Bonferroni s. In this section, we therefore discuss another analytical strategy for multiple comparisons, Tukey s honestly significant difference (HSD) test. 

SPSS 13.0 for Windows (1999-2004, SPSS Inc., Chicago, IL, USA) was used for data entry and statistical analysis. The experimental data obtained after the crossover dietary treatments were analyzed by using the repeated-measures analysis of variance (ANOVA). When a significant effect between dietary treatments was found, post hoc comparison of means was made using Bonferroni s test. Differences were considered statistically significant at P < 0.05. 

The major drawback of this method is that Bonferroni s Inequality is a conservative correction, especially if some of the hypotheses being tested are not independent. When many SNPs in the same gene are evaluated, for example, and are in LD with each other, the Boneferroni correction would not be appropriate, resulting in the possibility of false negatives or failure to detect a true association. A better approach would be to test the true level of significance directly through simulations. 

The practical consequence from this is that in the study type under consideration, always the dam/litter rather than the individual fetus is the basic statistical unit (see Chapters 23, 33, 34 and 35). Six malformed fetuses from six different litters in a treated group of dams is much more likely to constitute a teratogenic effect of the test substance than ten malformed fetuses all from the same litter. It is, therefore, important to report all fetal observations in this context and to select appropriate statistical tests (e.g., Fisher s exact test with Bonferroni correction) based on litter frequency. For continuous data, a procedure to calculate the mean value over the litter means (e.g., ANOVA followed by Dunnet s test) is preferred. An increase in variance (e.g., standard deviation), even without a change in the mean, may indicate that some animals were more susceptible than others, and may indicate the onset of a critical effect. 

The primary statistical tests used in the studies described in this text are based on the chi-square tests which are in turn derived from the chi-square distribution which is based on the chi distribution. These tests include the chi-square test for goodness of fit, the chi-square test of independence, and Fisher s Exact Test. There are also corrections to some of the tests that account for small number deviations, Yates Correction for Continuity, and for multiple studies attempting to verify the same procedures or processes, Bonferroni s correction. 

One-way ANOVA showedp < 0.0001 for each endpoint. Bonferroni s multiple comparison test showed significant difference p < 0.001) compared to vehicle-injected control... 

FIGURE 42.8. Ipsilateral cerebral F2-IsoPs (A) and F4-Neu-roPs (B) concentrations following i.c.v. KA with or without vitamin E (Vit E) or V-tert-butyl-a-phenylnitrone (PBN) pretreatment. Brains from mice exposed to KA were collected 30 min post-injections (n > 5 for each group). One-way ANOVA had p < 0.0001 with Bonferroni s multiple comparison tests significant for KA vs control, Vit E + KA or PBN + KA treatment. 

This means that the probability of rejecting at least one of c hypotheses is less than or equal to (thus the term "inequality") the sum of the probabilities of rejecting each hypothesis. This inequality is true even if the events, in this case rejecting one of c null hypotheses, are not independent. Recall from Section 6.2 that, when events are not independent, the probability of intersecting events should be subtracted. Using Bonferroni s method, testing each pair of means with an a level of a = will ensure that the overall type I error rate does not exceed the desired value of a. It follows that the probability of rejecting at least one of c null hypotheses can be expressed as follows ... 

When using Bonferroni s method, the null hypothesis associated with a pairwise comparison is rejected if the calculated test statistic, that is. 

The second step in our analytical strategy was to determine which of the pairs of means were significantly different from each other. Testing each of the three hypotheses at an a level of 0.05 would have resulted in a probability of committing a type I error possibly > 0.05 (the desired level). Bonferroni s inequality was therefore used to test each of the three hypotheses at an a level of 0.05/3 = 0.01667. Using the critical value for this a level resulted in two pairs of means being declared significantly different at the 0.05 level. 

Bonferroni s method for testing pairs of means (maintaining an overall type I error rate of a) involved comparing the absolute differences in means to the MSD, which was defined as a function of ... 

Having gone through the calculations necessary for Tukey s test, we can look at how these results would lead to decision-making, and also compare the interpretation and decision-making with those that followed from using Bonferroni s methodology on the same dataset. 

It should be noted that these are not the only acceptable methods applicable to multiple comparisons from an ANOVA. In each individual case, the choice among possible approaches is largely dependent on the study design. For example, Dunnett s test can be used when the only comparisons of interest are each test treatment versus a control (for example, in a placebo-controlled, dose-ranging study). Like Tukey s test, Dunnett s method is more powerful than Bonferroni s. In general, other methods gain power compared with Bonferroni s method by... 

FIGURE 16.3 Toosendanin (TSN) delays the time to 50% paralysis in isolated mouse hemidiaphragm muscle exposed to BoNT/A. TSN (3 jlM) was added to tissue baths 30 min before (pre), simultaneously with (simult.), or 30 min following addition of 20 pM BoNT/A (post). Paralysis times were significantly prolonged in muscles when TSN was added before or at the same time as BoNT/A (p < 0.01, ANOVA followed by Bonferroni post tests) as indicated by asterisks. Muscles tensions were elicited by supramaximal stimulation of the phrenic nerve at 30 s intervals. Temperature, 37°C. 

Fig. 13. Dose-related effects of EM574 top), motilin middle), and cisapride bottom) on gastric antral motor index left) and gastric emptying indicated as plasma acetaminophen concentration right). EM574 and cisapride were administered intraduodenally immediately after the meal. Motilin was infused intravenously during 30-min postprandial period. Data are expressed as means SE of five dogs. E < 0.05, E < 0.01 compared to each control by paired t test with Bonferroni s correction. (From Sato et al. [22] with permission from Elsevier Science.)...

Statistics. One-way analysis of variance and Bonferroni s post hoc tests were used to compare values of half-life and systemic clearance for ATI and ATF. Differences were considered to be significance when p < 0.05. 

It is possible to construct procedures which simultaneously test the null hypotheses that a number of outcomes are identical for the two treatments. Such tests can use the empirical correlation structure of the observations and thus deal with the third of the criticisms of the Bonferroni approach. An example of such a test is Hotelling s test. 

Laboratory and/or field data were analyzed using SAS systems (SAS Institute Inc. 1999) utilizing analysis of variance, regression analysis, response surface analysis, univariate analysis, repeated measures analysis (multivariate profile analysis), covariance analysis and/or principle components analysis. Good statistical practices were used to verify that the data satisfied the assumptions underlying the various analyses. Significant differences between means were determined by Tukey s Studentized Range Test, the Tukey-Kramer HSD test, or the Bonferroni t test. Alpha was set at 0.05. 

---