Neyman-Pearson hypothesis testing

This brief excursion Into Decision Theory Is Included to Indicate the manner In which experimental data can be coupled with external (societal) judgments to form a logical basis for societal decisions and actions. A justification for so complex a strategy for decision making is that "simple scientific measurements and model evaluations will always be characterized by measurement uncertainty. Yet societal decisions and actions must take place even under the shadow of uncertainty. For scientific measurements, as discussed in Che following text, however, we shall restrict our attention to the relatively simple Neyman-Pearson hypothesis testing model (8, p. 198). 

The measurement begins with Neyman-Pearson type of testing(gL> of the following null hypothesis ... 

Analytical methods are not ordinarily associated with the Neyman-Pearson theory of hypothesis testing. Yet, statistical hypothesis tests are an indispensable part of method development, validation, and use Such testa are used to construct analytical curves, to decide the "minimum significant measured" quantity, and the "minimum detectable true" quantity (33.34) of a method, and in handling the "outlier value problem"(35.36). 

The hypothesis testing framework described above is basically the system proposed by J. Neyman and E. Pearson (see Chapter 2). It can be regarded as the official modern frequentlst approach to clinical trials. In practice, an older approach due to R.A. Fisher (see Chapter 2) is often used. 

The second practical (and semi-philosophical) objection to the Neyman-Pearson approach is that it is often more natural to define the test statistic right away rather than the alternative hypothesis. In fact, most of the more common standard tests had already been derived by R.A. Fisher without reference to alternative hypotheses before the Neyman-Pearson theory was developed. Thus the posited alternative hypothesis in these formulations might unkindly be described as being an excuse for the test statistic adopted, rather than a reason. 

Nevertheless, it is fair to say that there are some more complicated cases where it is perhaps easier to define the alternative hypothesis than the test and that the Neyman-Pearson theory then provides a way in which a test can be derived. 

In the Neyman-Pearson theory of hypothesis testing, the nature of the alternative hypothesis helps to define the test. Let t be the true difference between active and placebo. According to this theory, if we can write the following as our null and alternative hypotheses. 

If we wish to say something about the difference which obtains, then it is better to quote a so-called point estimate of the true treatment effect, together with associated confidence limits. The point estimate (which in the simplest case would be the difference between the two sample means) gives a value of the treatment effect supported by the observed data in the absence of any other information. It does not, of course, have to obtain. The upper and lower 1 — a confidence limits define an interval of values which, were we to adopt them as the null hypothesis for the treatment effect, would not be rejected by a hypothesis test of size a. If we accept the general Neyman-Pearson framework and if we wish to claim any single value as the proven treatment effect, then it is the lower confidence limit, rather than any value used in the power calculation, which fulfills this role. (See Chapter 4.)... 

As Talias (2007) has pointed out, there is an interesting analogy between the Pearson Index and the Neyman-Pearson lemma. (The Pearsons in question are different. Alan Pearson is the author of the Pearson index and Egon Pearson, 1895-1980, was the son of Karl Pearson, 1857-1936 and the collaborator of Jerzey Neyman, 1894-1981, in developing hypothesis testing.) Both are relevant to optimizing a function subject to a constraint. In the case of the Pearson index this is profit subject to total cost, and for the Neyman-Pearson lemma it is power subject to the constraint of an overall type I error rate. In both cases a ratio plays a key role. For the Neyman-Pearson lemma this is the likelihood ratio and for the Pearson index the index itself is a ratio of expected profit to expected cost. 

Detection of signals in the presence of noise, using classical Bayes or Neyman Pearson decision criteria, is based on hypothesis testing. In the simplest binary hypothesis case, the detection problem is posed as two hypotheses ... 

---