Hypothesis testing General considerations

As this book focuses on clinical trials our primary interest is in providing you with relevant examples of hypothesis testing in that arena. However, it is useful initially to lay some conceptual foundations with simpler examples. As for many other examples in statistics and probability, we illustrate these concepts first with flips of a coin. 

Suppose you then revise the experiment and request that the results from 10 flips of the coin be recorded. You reason that, if the coin were fair, you would expect five heads and five tails. If you were to observe that only one or as many as nine heads came up out of ten tosses, you would conclude that the coin was not fair. Your logic is that, by chance alone, a fair coin would not very likely yield such a lopsided result. If you were to observe an event with even more extreme result, that is, 0 or 10 heads out of 10 tosses, you would also have concluded, perhaps with even more confidence, that the coin was not fair. 

The rule that you intuitively arrived at was that if you observed as few as 0 or 1 or as many as 9 or 10 heads out of 10 coin flips, you would conclude that the coin was not fair. How likely is it that such a result would happen In other words, suppose you repeated this experiment a number of times with a truly fair coin. What proportion of experiments conducted in the same manner would result in an erroneous conclusion on your part because you followed the evidence in this way This is the point where the rules of probability come into play. You can find the probability of making the wrong conclusion (calling the fair coin biased) by 

The process of testing a hypothesis usually begins with the statement of the hypothesis that we would like to conclude as a result of the research (we refer to this as the alternate 

The next part of the hypothesis-testing process is to decide on a numericai resuit (a test statistic) that, if observed, wouid sufficiency contradict the nuii hypothesis such that the nuii hypothesis wouid be rejected in favor of the alternate hypothesis. As we discovered with our cointossing example, some results would not be all that rare by chance alone. Therefore, our decision rule should be defined such that erroneous conclusions are not made more often than we are willing to tolerate. 

---

The aforementioned probability-based considerations may serve as the most important foundations for derivation of statistically assured decisions. In general, in inferential statistics, the first step is the statement of a hypothesis, the significance of which is tested against a given risk a. 

It is a good idea to know how many units (i.e., items or subjects) are needed to test our hypothesis. Unless we test all the units in a population, we are only testing a sample of the whole population. To make a general conclusion about the population, we need to show that the effect that we observed was not likely the result of chance from random variation in our sample. If we choose too few units, we may end up with an inconclusive result and a worthless study, and if we choose too many, we are wasting resources (e.g., sacrificing more animals than needed). Hence, sample size consideration is of ethical relevance (Bacchetti et al., 2005). 

---