Statistical significance and clinical importance

Trial 1 has given statistical significance and has detected something of clinical importance. 

Drummond MF, O Brien B. 1993. Clinical importance, statistical significance and the assessment of economic and quality of life outcomes. Health Econ. 2 205. 

Geriatric Considerations - Summary Sink and colleagues systematic review showed statistically significant improvements on neuropsychiatric and behavioral scales for some drugs, but improvements were small and unlikely to be clinically important. Because of documented risks and uncertain benefits, these agents should be used with caution in demented elderly persons, with frequent monitoring for side effects and a low threshold for discontinuing use. 

This choice should allow, amongst other things, a clear quantitative measure of benefit at the individual patient level. As we will see, identifying new treatments is not just about statistical significance, it is also about clinical importance and the... 

Trial 3 has given non-significance statistically and inspecting the confidence interval tells us that there is nothing in terms of clinical importance either at most with 95 per cent confidence the benefit of the active treatment is only 3.1 mmHg. 

The regulators are not only interested in statistical significance but also in clinical importance. This allows them, and others, to appropriately balance benefit and risk. It is good practice therefore to present both p-values and confidence intervals and indeed this is a requirement within a submission. Most journals nowadays also require results to be presented in the form of confidence intervals in addition to p-values. 

The clinical significance of these CT scan alterations is unknown, and it is important to note that two similar studies found no statistically significant differences between the CT scans of long-term BZD users and matched control subjects ( 212, 213). Poser et al. (212) did find, however, that patients with combined BZD—alcohol dependence showed some degree of cerebral atrophy. Uhde and Kellner (211) also observed that the apparent association between VBR and duration of BZD exposure may be secondary to alcohol consumption, because long-term BZD users also may be more frequent users, but not necessarily abusers, of alcohol. 

---