Stepdown testing

Stepdown Testing—A Shortcut to Closed and Partition Testing... 

The key condition needed to effect a shortcut is roughly that the rejection of a more restrictive hypothesis implies the rejection of certain less restrictive null hypotheses. So if one starts by testing the more restrictive null hypothesis and then skips the testing of less restrictive hypotheses as such implications allow, then closed or partition testing becomes more computationally feasible. The resulting shortcut version of a closed or partition test is a stepdown test. 

A method of this form is called a stepdown test because it steps down from the most statistically significant to the least statistically significant. 

One should verify either conditions SI to S3 are satisfied, or subset pivotality is satisfied, before implementing a stepdown test for, otherwise, the stepdown test may not strongly control the familywise error rate. Such conditions are easier to check with a model that connects the observations with the parameters, but harder to check with a model (such as the randomization model) that only describes the distribution of the observations under the null hypotheses. Indeed, Westfall and Young (1993, page 91) cautioned that the randomization model does not guarantee that the subset pivotality condition holds. Outside the context of bioinformatics, there are in fact examples of methods that were in use at one time that violate... 

For this data set, at a familywise error rate of 5%, one-step testing leads to the conclusion that 174 genes are differentially expressed. Closed and partition stepdown testing are guaranteed to infer at least those same 174 genes to be differentially expressed. In fact, these last two methods both infer three additional genes to be differentially expressed that is, a total of 177 genes. 

Another condition, similar to conditions SI to S3, which also guarantees that a stepdown method strongly controls the family wise error rate is the subset pivotality condition proposed by Westfall and Young (1993, page 42). Their original subset pivotality condition is given in terms of adjusted -values. For comparability with S1 to S3, we have paraphrased that condition here in terms of test statistics ... 

---