Bonferroni test

Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. 

Another approach to multiple comparisons is the Bonferroni test. The strategy in this case is to divide a (typically 0.05 when a single comparison is being made) by the number of tests conducted following a significant omnibus ANOVA test. Hence, if ten comparisons were to be made, the a-level used for each comparison would become 0.05/10, i.e., 0.005, a considerably more conservative value. 

Fig. 2.20 Development of the hydroperoxide content of microencapsulated fish oil prepared with native and fibrillar WPI at pH 2.0 and 3.0 during storage at 20 °C and 33 % rh (dark). Data are means from triplicate analysis of two independently stored samples. Coefficient of variation between independent samples ranged from 0.1 to 4.5%. Values with different letters are significantly different in Bonferroni Test (p < 0.05) at a given storage time. Reproduced with permission from [56]...

The major drawback of this method is that Bonferroni s Inequality is a conservative correction, especially if some of the hypotheses being tested are not independent. When many SNPs in the same gene are evaluated, for example, and are in LD with each other, the Boneferroni correction would not be appropriate, resulting in the possibility of false negatives or failure to detect a true association. A better approach would be to test the true level of significance directly through simulations. 

It tests all linear contrasts among the population means (the other three methods confine themselves to pairwise comparison, except they use a Bonferroni type correlation procedure). 

The practical consequence from this is that in the study type under consideration, always the dam/litter rather than the individual fetus is the basic statistical unit (see Chapters 23, 33, 34 and 35). Six malformed fetuses from six different litters in a treated group of dams is much more likely to constitute a teratogenic effect of the test substance than ten malformed fetuses all from the same litter. It is, therefore, important to report all fetal observations in this context and to select appropriate statistical tests (e.g., Fisher s exact test with Bonferroni correction) based on litter frequency. For continuous data, a procedure to calculate the mean value over the litter means (e.g., ANOVA followed by Dunnet s test) is preferred. An increase in variance (e.g., standard deviation), even without a change in the mean, may indicate that some animals were more susceptible than others, and may indicate the onset of a critical effect. 

The primary statistical tests used in the studies described in this text are based on the chi-square tests which are in turn derived from the chi-square distribution which is based on the chi distribution. These tests include the chi-square test for goodness of fit, the chi-square test of independence, and Fisher s Exact Test. There are also corrections to some of the tests that account for small number deviations, Yates Correction for Continuity, and for multiple studies attempting to verify the same procedures or processes, Bonferroni s correction. 

Note that performing multiple hypothesis tests (e.g., Student s t-test) may inflate the false positive rate. That is, the number of genes detected as active by chance alone will increase with the number of genes tested. For example, a microarray with 7000 features would require at least 7000 hypothesis tests per treatment comparison. Several methods have been developed to control the false positive rate, such as the conservative Bonferroni correction and the FDR control method (49). 

Bonferroni keeps the rate of type I errors down to 5 percent... In many instances, omnibus tests are not possible. If there are several comparisons to be made, each involving different groups of individuals and different endpoints, there may be no choice but to use a series of discrete statistical tests. In these circumstances, the Bonferroni correction may be applied to maintain the overall risk of false positives at the standard level of 5 per cent. 

What the Bonferroni correction does is to raise the standard of proof for all the individual tests. Each test is then less likely to produce a false positive and the complete series of analyses will jointly generate a 5 per cent risk. 

The Bonferroni correction raises the standard of proof required for each individual test. This has the effect of maintaining the overall risk of any false positives at 5 per cent, but reduces statistical power. 

Previous discussions of multiplicity adjusted testing of gene expressions, by Dudoit et al. (2002) and (2003), for example, generally took a nonmodeling approach. Because the joint distribution of the test statistics is generally not available with this approach, multiplicity adjustments in these papers tend to be calculated based on conservative inequalities (for example, the Bonferroni inequality or Sidak s inequality) or on a joint distribution of independent test statistics. In contrast, here, we describe multiplicity adjustment based on the actual joint distribution of the test statistics. However, before describing such adjustments, we first address the construction principles to which all multiple tests should adhere, regardless of the approach taken. These principles do not appear to be as well known in the field of bioinformatics as they are in clinical trials. 

One-way ANOVA showedp < 0.0001 for each endpoint. Bonferroni s multiple comparison test showed significant difference p < 0.001) compared to vehicle-injected control... 

FIGURE 42.8. Ipsilateral cerebral F2-IsoPs (A) and F4-Neu-roPs (B) concentrations following i.c.v. KA with or without vitamin E (Vit E) or V-tert-butyl-a-phenylnitrone (PBN) pretreatment. Brains from mice exposed to KA were collected 30 min post-injections (n > 5 for each group). One-way ANOVA had p < 0.0001 with Bonferroni s multiple comparison tests significant for KA vs control, Vit E + KA or PBN + KA treatment. 

---