Confidence intervals clinical importance

Trial 3 has given non-significance statistically and inspecting the confidence interval tells us that there is nothing in terms of clinical importance either at most with 95 per cent confidence the benefit of the active treatment is only 3.1 mmHg. 

Trial 4 is different, however. We do not have statistical significance, but the confidence interval suggests that there could still be something of clinical importance with potential differences of 5 mmHg, 10 mmHg, even... 

The regulators are not only interested in statistical significance but also in clinical importance. This allows them, and others, to appropriately balance benefit and risk. It is good practice therefore to present both p-values and confidence intervals and indeed this is a requirement within a submission. Most journals nowadays also require results to be presented in the form of confidence intervals in addition to p-values. 

It is all too common to see a conclusion that treatments are the same (or similar) simply on the back of a large p-value this is not necessarily the correct conclusion. Presentation of the 95 per cent confidence interval will provide a statement about the possible magnitude of the treatment difference. This can be inspected and only then can a conclusion of similarity be made if this interval is seen to exclude clinically important differences. We will return to a more formal approach to this in Chapter 12 where we will discuss equivalence and non-inferiority. 

An important question in meta-analysis is the consistency of the results (i.e., its confidence interval). Thus, we not only want to know how much more effective a drug is but whether all the clinical trials agree on the size of the therapeutic effect. 

Presentation of the confidence interval facilitates evaluation of the reported data in original measurement units, which simultaneously incorporates the statistical changes observed and the precision of the measurements in the units in which the data were measured. The confidence interval provides a bridge to clinical importance, in that the reader can see the observed experimental change in the context of a range of uncertainty in the same units. Indeed, for this reason, the reporting of confidence intervals is required by many periodicals. 

Two common statistical techniques that are typically used to analyze efficacy data in superiority trials are f-tests and ANOVA. In parallel group trials, the independent groups Mest and the independent groups ANOVA discussed in Chapter 7 would be used. Another important aspect of the statistical methodology employed in superiority trials, the use of CIs (confidence intervals) to estimate the clinical significance of a treatment effect, was discussed in Chapter 8. These discussions are not repeated here. Instead, some additional aspects of statistical methodology that are relevant to superiority trials are discussed. 

Confidence intervals are extremely informative in clinical research since they do focus on the estimated treatment effect and therefore facilitate consideration of its clinical significance. As Fletcher and Fletcher (2005) noted succinctly and powerfully, confidence intervals put the emphasis where it belongs, on the size of the effect. The width of a confidence interval around an experimentally determined treatment effect, and hence the range of plausible values for the population treatment effect, provides very important information about the clinical significance of the treatment. 

As we saw in Chapter 8, confidence intervals can be used to deduce levels of statistical significance While they do not yield precise p-values, they reveal whether or not a given level of statistical significance is achieved. More importantly, they are uniquely informative in assessing clinical significance. Therefore, confidence intervals offer a tremendous advantage over p-values in the clinical context, and they... 

Stacistical signincance and its clinical importance Confidence intervals... 

There are a number of values of the treatment effect (delta or A) that could lead to rejection of the null hypothesis of no difference between the two means. For purposes of estimating a sample size the power of the study (that is, the probability that the null hypothesis of no difference is rejected given that the alternate hypothesis is true) is calculated for a specific value of A. in the case of a superiority trial, this specific value represents the minimally clinically relevant difference between groups that, if found to be plausible on the basis of the sample data through construction of a confidence interval, would be viewed as evidence of a definitive and clinically important treatment effect. 

Moreover, the strength of current signal detection systems is in detecting new or unexpected findings, often relating to more extreme phenotypes of adverse reactions that have low background incidences. This gives us only a small snapshot of the beast, and we are seldom able to derive clinically important measures, such as the relative risk and its confidence intervals, or absolute estimates of risk, such as the likely number of patients affected or the number needed to treat for harm (NNTh). 

---