Clinical trials confidence intervals

Trial 3 has given non-significance statistically and inspecting the confidence interval tells us that there is nothing in terms of clinical importance either at most with 95 per cent confidence the benefit of the active treatment is only 3.1 mmHg. 

Trial 4 is different, however. We do not have statistical significance, but the confidence interval suggests that there could still be something of clinical importance with potential differences of 5 mmHg, 10 mmHg, even... 

Clearly the main advantage of a non-parametric method is that it makes essentially no assumptions about the underlying distribution of the data. In contrast, the corresponding parametric method makes specific assumptions, for example, that the data are normally distributed. Does this matter Well, as mentioned earlier, the t-tests, even though in a strict sense they assume normality, are quite robust against departures from normality. In other words you have to be some way off normality for the p-values and associated confidence intervals to be become invalid, especially with the kinds of moderate to large sample sizes that we see in our trials. Most of the time in clinical studies, we are within those boundaries, particularly when we are also able to transform data to conform more closely to normality. 

In a clinical trial with the objective of demonstrating non-inferiority, suppose that the data are somewhat stronger than this and the 95 per cent confidence interval is not only entirely to the right of — A, but also completely to the right of zero as in Figure 12.5 there is evidence that the new treatment is in fact superior. 

An important question in meta-analysis is the consistency of the results (i.e., its confidence interval). Thus, we not only want to know how much more effective a drug is but whether all the clinical trials agree on the size of the therapeutic effect. 

This chapter introduces basic concepts in statistical analysis that are of relevance to describing and analyzing the data that are collected in clinical trials, the hallmark of new drug development. (Statistical analysis in nonclinical studies was addressed earlier in Chapter 4.) This chapter therefore sets the scene for more detailed discussion of the determination of statistical significance via the process of hypothesis testing in Chapter 7, evaluation of clinical significance via the calculation of confidence intervals in Chapter 8, and discussions of adaptive designs and of noninferiority/equivalence trials in Chapter 11. 

Two common statistical techniques that are typically used to analyze efficacy data in superiority trials are f-tests and ANOVA. In parallel group trials, the independent groups Mest and the independent groups ANOVA discussed in Chapter 7 would be used. Another important aspect of the statistical methodology employed in superiority trials, the use of CIs (confidence intervals) to estimate the clinical significance of a treatment effect, was discussed in Chapter 8. These discussions are not repeated here. Instead, some additional aspects of statistical methodology that are relevant to superiority trials are discussed. 

Hsu, J. C. (1996). Multiple Comparisons Theory and Methods. Chapman Hall, London. Hsu, J. C., Chang, J. Y., and Wang, T. (2002). Simultaneous confidence intervals for differential gene expressions. Technical Report 592, The Ohio State University, Columbus. ICH E10 (1999). Choice of Control Groups in Clinical Trials. CPMP (Committee for Pro-pritary Medical Products), EMEA (The European Agency for the Evaluation of Medical Products), London, Draft ICH (International Conference on Harmonisation). Efficiency guidelines, http //www.ich.org. 

Fergusson et al. (2005) searched the literature and found 702 randomized clinical trials (87,650 patients) comparing SSRIs with either placebo or an active non-SSRI control medication. They found a statistically significant, more than two-fold increased risk of suicide attempts on SSRIs compared to placebo. The odds ratio of suicide attempts in SSRI-treated patients versus placebo patients was 2.28 (p = 0.02) and a 95% confidence interval (Cl) of 1.14-4.55. They also found an increased suicide risk between SSRIs and other medications, excluding tricyclic antidepressants. There was no difference between the SSRIs and tricyclics in suicide risk. Overall, their results documented an association between suicide attempts and the use of SSRIs. ... 

Earlier, this chapter reviewed Healy et al. s (2006) finding that clinical trials in paroxetine for children found an increased number of hostile events and that the rates were highest in children with obsessive compulsive disorder (OCD), where the odds ratio of a hostile event was 17 times greater (95% confidence interval [Cl], 2.22-130.0). ... 

The key to the ethics of such studies is informed consent from patients, efficient scientific design and review by an independent research ethics committee. The key interpretative factors in the analysis of trial results are calculations of confidence intervals and statistical significance.The potential clinical significance needs to be considered within the confines of controlled clinical crials.This is best expressed by stating not only the percentage differences, but also the absolute difference or its reciprocal, the number of patients who have to be treated to obtain one desired outcome.The outcome might include both efficacy and safety... 

Randomised controlled trials with definitive results (confidence intervals that do not overlap the threshold of the clinically significant effect)... 

Randomised controlled trials with nondefinitive results (a difference that suggests a clinically significant effect but with confidence intervals overlapping the threshold of this effect)... 

Another useful interpretation of confidence intervals is that the values that are enclosed within the confidence interval are those that are considered the most plausible values of the unknown population parameter. Values outside the interval are considered less plausible. All other things being equal, the need for greater confidence in the estimate results in wider confidence intervals, and confidence intervals become narrower (that is, more precise) as the sample size increases. This last fact is explored in greater detail in Chapter 12 because it is directly relevant to the estimation of the required sample size for a clinical trial. The methods to use for the calculation of confidence intervals for other population parameters of interest are provided in subsequent chapters. 

In Chapter 6 we described the basic components of hypothesis testing and interval estimation (that is, confidence intervals). One of the basic components of interval estimation is the standard error of the estimator, which quantifies how much the sample estimate would vary from sample to sample if (totally implausibly) we were to conduct the same clinical study over and over again. The larger the sample size in the trial, the smaller the standard error. Another component of an interval estimate is the reliability factor, which acts as a multiplier for the standard error. The more confidence that we require, the larger the reliability factor (multiplier). The reliability factor is determined by the shape of the sampling distribution of the statistic of interest and is the value that defines an area under the curve of (1 - a). In the case of a two-sided interval the reliability factor defines lower and upper tail areas of size a/2. 

Within-group confidence intervals can be informative, but usually the primary interest in a clinical trial is to compare the effect of one treatment with that of another. Therefore, a confidence interval for the difference in two means can better address the goals of the research. 

There are a number of values of the treatment effect (delta or A) that could lead to rejection of the null hypothesis of no difference between the two means. For purposes of estimating a sample size the power of the study (that is, the probability that the null hypothesis of no difference is rejected given that the alternate hypothesis is true) is calculated for a specific value of A. in the case of a superiority trial, this specific value represents the minimally clinically relevant difference between groups that, if found to be plausible on the basis of the sample data through construction of a confidence interval, would be viewed as evidence of a definitive and clinically important treatment effect. 

---