Statistical/clinical relevance

Methods that select covariates by choosing those that are most strongly associated with the primary outcome (often called variable selection methods ) should be avoided. The clinical and statistical relevance of a covariate should be assessed and justified from a source other than the current dataset. ... 

Up to now, none of the presented system can claim its ability to fully replace all liver functions in an extracorporeal circuit. On the one hand, purely artificial techniques can only cover some detoxification aspects, which is already crucial in many clinical cases to save patients. On the other hand, bioartificial livers have not proven their full efficiency yet, mainly because both regulatory and logistic aspects limit, for the moment, the inclusion of significant numbers of patients to draw statistically relevant conclusions. 

The statisticians conducting the review of an NDA evaluate the statistical relevance of the data presented so that they can provide the medical officers with information concerning how well the findings are likely to generalize to the larger patient population in the country. They evaluate the extent of any deviations from the protocols submitted in the IND in the conduct of the study as well as the overall quality of the data collected. All clinical study protocol amendments are reviewed to see what deviations from the original study design have occurred and how these (and deviations that were not detailed in protocol amendments) may have influenced the data. 

Information regarding the relative value of treatments is often accumulated slowly. Hence, interim analysis is important. This means that results of a clinical trial are analyzed while the study is still in progress. When the results become statistically relevant (which may happen much earlier than predicted in the study protocol), the new treatment may prove beneficial or undesirable. In such a case ethical considerations require the trial to be concluded, in order that all patients should receive the more efficient treatment. 

A key element in planning and conducting clinical trials is to ensure that they have scientific validity and objectivity. This is particularly relevant with respect to Phase II and III studies, where it is desired to demonstrate a positive benefit to risk outcome. Responses to a drug among a patient population are rarely homogeneous and clear-cut. Thus, sound statistical principles must be applied in order to be able to distinguish significant effects from random events. 

Interpretation. The process whereby one determines the clinical meaning or significance of data after the relevant statistical analyses have been performed. These processes often involve developing an explanation of the data that are being evaluated. 

In Figure 8.9, we illustrate various cases that can arise from studies intended to show equivalence and the relationship between significance in the traditional sense and clinical significance as determined by the confidence interval and the boundaries of equivalence. In case (A), the 95% confidence interval includes both the null hypothesis of no difference and is within the boundaries of equivalence and from both a statistical and clinical perspective there is no evidence of a difference between the treatments. In case (B), in contrast, the confidence interval is still within the boundaries, but does include the null hypothesis, so from a statistical perspective there is a difference between the treatments but it is not clinically relevant. Case (C) shows both statistical and clinical significance, as the confidence interval lies outside the equivalence boundaries and therefore cannot include the null hypothesis. In the final case, (D), the confidence interval includes... 

Intermittent claudication appears to benefit from ginkgo therapy many studies demonstrate improved walking distance and decreased pain. One meta-analysis of eight studies documented statistically significant improvement but questioned its clinical relevance. In some studies, the high doses (240 mg) appeared more effective. 

Clearly it is very important that we get the hierarchy correct. Generally this would be determined by the clinical relevance of the endpoints, although under some circumstances it could be determined, in part, by the likelihood of seeing statistical significance with the easier hits towards the top of the hierarchy. 

While thep-value allows us the ability to judge statistical significance, the clinical relevance of the finding is difficult to evaluate from the calculated confidence interval because this is now on the log scale. It is usual to back-transform the lower and upper confidence limits, together with the difference in the means on the log scale, to give us something on the original data scale which is more readily interpretable. The back-transform for the log transformation is the anti-log. 

A difference can be statistically significant but have no clinical relevance. Statistical significance is determined... 

Antipsychotics versus psychotherapy (two comparative studies). Both studies covered large, statistically equivalent groups of patients with productive symptoms one study compared group psychotherapy and antipsychotics the other compared analytically orientated individual therapy and antipsychotics. The results were similar inasmuch as both trials showed the drug therapy to be markedly superior in almost all clinically relevant characteristics and symptoms. 

---