Family-wise error rate

Another condition, similar to conditions SI to S3, which also guarantees that a stepdown method strongly controls the family wise error rate is the subset pivotality condition proposed by Westfall and Young (1993, page 42). Their original subset pivotality condition is given in terms of adjusted -values. For comparability with S1 to S3, we have paraphrased that condition here in terms of test statistics ... 

We touched on this problem in Chapter 9, where we drew attention to Cournot s criticism of multiple comparisons. To use the language of hypothesis testing, the problem is that as we carry out more and more tests, the probability of making at least one type I error increases. This probability of at least one type I error is sometimes referred to as the family-wise error rate (FWER) (Benjamini and Hochberg, 1995). Thus, controlling the type I error rates of individual tests does not guarantee control of the FWER. To put... 

Now, suppose all null hypotheses are true, so that Mq = m, then m — Wq = 0 and therefore 5=0 and testing each hypothesis at level a will give E [V jm = a. In general, if there are some false null hypotheses, then the expected value of V will be less but the value of m will be unchanged so that the E [V jm < a. Thus the PCER is controlled at or below a by this strategy. On the other hand, the family-wise error rate (FWER) is simply the probability that at least one hypothesis is falsely rejected, and this is P (V > 1). This... 

---