Testing non-inferiority

If your target journal sends your paper to a statistically savvy reviewer, you will not get away with this one, but do not worry - many of them do not. 

Your new treatment or analytical method (or whatever) is supposed to produce the same results as the old one, but the wretched thing obviously does not. Do not despair, just follow these simple instructions  

The failure to find a difference, does not necessarily imply that none exists. Maybe you did not look very hard. 

With equivalence testing there are two possibilities that we need to dispose of - the possibilities that the difference between outcomes might be either (a) greater than some upper limit or (b) less than a lower limit. The propranolol tablets provide an 

Non-inferiority testing is used to see whether a product/process is at least as good as some alternative. 

---

Figure 9.5 Interpretation of non-inferiority testing. Comparison of new dialysis method with old. Difference in plasma urea concentration (mM)...

Notice the difference. For equivalence testing we have to show that any change lies between an upper and a lower limit, but with non-inferiority testing we only have to show that there is no change beyond a single (upper or lower) limit. 

Demonstrating non-inferiority To demonstrate that a procedure or treatment is at least as good as an alternative (without needing to show that it is any better) requires non-inferiority testing. We calculate a one-sided 95 per cent confidence limit for the difference between the two treatments. The limit calculated is the one that indicates the greatest credible deterioration associated with the new procedure, ff this limit precludes any realistic possibility of a deterioration large enough to matter, we may claim evidence of non-inferiority. 

Practically significant change, equivalence and non-inferiority testing... 

As we saw in Chapter 9, each of these questions will lead us into a distinctive statistical approach - difference testing, equivalence testing or non-inferiority testing. 

In the case of difference testing, the other question that needs to be settled at a very early stage is whether it would be sufficient simply to know whether a difference exists, or is the size of difference an issue In some cases the former may be adequate and a simple P value will settle the issue, but where the size of difference is important, the 95 per cent Cl for difference must be inspected (see Chapter 9). For equivalence or non-inferiority testing, only the 95 per cent Cl is of the slightest use. 

Fig. 6.1 Relationship between significance tests and confidence intervals for the comparison between a new treatment and control. The treatment differences A and B are in favour of the new treatment but superiority is shown only in A. in B, the outcome may meet criteria for equivalence or non-inferiority as defined in the protocol.

If in Figure 8.9 a positive difference between treatments were indicative of a benefit for the test treatment then case (C) would indicate significant superiority of the new treatment. In such circumstances, we would not wish to conclude that only the treatments were not equivalent. In such circumstances, we can use a single boundary and such studies are called non-inferiority studies in which the objective is to show that the new treatment is no more than a small amount worse than the standard. The conduct of the inference remains similar if the confidence interval is to the right of the non-inferiority boundary, we can conclude that the new treatment is non-inferior to the standard. 

In therapeutic equivalence trials and in non-inferiority trials we are often looking to demonstrate efficacy of our test treatment indirectly. It may be that for ethical or practical reasons it is not feasible to show efficacy by undertaking a superiority trial against placebo. In such a case we compare our test treatment to a control treatment that is known to be efficacious and demonstrate either strict... 

In Figure 12.2, and P2 e mean reductions in diastolic blood pressure in the test treatment and active control groups respectively. If the difference in the means is above zero then the test treatment is superior to the active control, if the difference is zero then they are identical. If the difference falls below zero the test treatment is not as good as the active control. This, however, is a price we are prepared to pay, but only up to a mean reduction in efficacy of 2 mmHg beyond that, the price is too great. The non-inferiority margin is therefore set at —2 mmHg. 

Step 2 is then to run the trial and compute the 95 per cent confidence interval for the difference, Pi — P2> in the mean reductions in diastolic blood pressure. In the above example suppose that this 95 per cent confidence interval turns out to be ( — 1.5 mmHg, 1.8 mmHg). As seen in Figure 12.2, all of the values within this interval are compatible with our definition of non-inferiority the non-inferiority of the test treatment has been established. In contrast, had the 95 per cent confidence interval been, say, (—2.3 mmHg,... 

The aim of this study reported by Powderly et al. (1992) was to establish the non-inferiority of a test treatment, fluconazole, compared to an established treatment, amphotericin B, in preventing the relapse of cryptococcal meningitis in HIV-infected patients. It was thought that fluconazole would be less effective than amphotericin B, but would offer other advantages in terms of reduced toxicity and ease of administration fluconazole was an oral treatment while amphotericin B was given intravenously. The non-inferiority margin was set at —15 per cent in terms of relapse rates. 

Using a 2.5 per cent significance level for non-inferiority in this way may initially appear to be out of line with the conventional 5 per cent significance level for superiority. A moments thought should suffice however, to realise that in a test for superiority we would never make a claim if our treatment was significantly worse than placebo, we would only ever make a claim if we were significantly better than placebo, so effectively we are conducting a one-sided test at the 2.5 per cent level to enable a positive conclusion of superiority for the active treatment. 

We will focus our attention to the situation of non-inferiority. Within the testing framework the type I error in this case is as before, the false positive (rejecting the null hypothesis when it is true), which now translates into concluding noninferiority when the new treatment is in fact inferior. The type II error is the false negative (failing to reject the null hypothesis when it is false) and this translates into failing to conclude non-inferiority when the new treatment truly is non-inferior. The sample size calculations below relate to the evaluation of noninferiority when using either the confidence interval method or the alternative p-value approach recall these are mathematically the same. 

In an anti-infective non-inferiority study it is expected that the true cure rates for both the test treatment and the active control will be 75 per cent. A has been chosen to be equal to 15 per cent. Using the usual approach with a one-sided 97.5 per cent confidence interval for the difference in cure rates a total of 176 patients per group will give 90 per cent power to demonstrate non-inferiority. Table 12.1 gives values for the sample size per group for 90 per cent power and for various departures from the assumptions. 

When the cure rates are equal, the sample size decreases as the common cure rate increases. When the test treatment cure rate is above the active control cure rate then the test treatment is actually better than the active control and it is much easier to demonstrate non-inferiority. When the reverse happens, however, where the test treatment cure rate falls below that of the active control, then the sample size requirement goes up it is much more difficult under these circumstances to demonstrate non-inferiority. It is also worth noting that when the test treatment is truly 15 per cent, the value for A in this example, below the rate in the active control, demonstrating non-inferiority is simply not possible. 

A finding of no significant difference in outcome between two treatments is ambiguous unless a non-inferiority design is used. Two-arm non-inferiority designs require much larger sample sizes to test adequately the statistical hypothesis of no true difference in efficacy between a test and reference drug. 

---