Meta-analysis context

With regard to relevant statistical methodologies, it is possible to dehne 2 situations, which can be termed a meta-analysis context and a shrinkage estimation context. Similar statistical models, in particular random-effects models, may be applicable in both situations. However, the results of such a model will be used somewhat differently. 

Egger M, Davey Smith G, Altman DG, eds. Systematic Reviews in Healthcare Meta-analysis in Context, 2nd edn. London BMJ Books, 2001. 

There is some USEPA precedent for use of statistical meta-analysis in a regulatory context, including the recent meta-analysis of organophosphate-related acetylcholinesterase inhibition data and meta-analysis of epidemiological studies on effects of 2nd hand tobacco smoke exposure. Warren-Hicks and Moore (1998) provide some discussion of the potential applicability of meta-analysis to ecological risk assessments. 

The essence of meta-analysis is inspection of the data. Thus, this approach produces a visual or numeric representation of each study in the context of all the others. A review of the actual data gives the critical reader a feel for the data, as well as an index of suspicion if there is undue variability, which is far more important than any statistical parameter. 

In this context, Lilly (474) reported a meta-analysis of three controlled studies of patients with TD who were treated with olanzapine. These authors found an 11-fold decrease in TD on olanzapine versus haloperidol based on the AIMS scale. There were a few patients who developed TD in the first 6 weeks of olanzapine, but this could have been from previous drug exposure, now not suppressed by the neuroleptic. Interestingly, there were no new cases (0/375) of TD developing in patients on long-term olanzapine treatment, whereas there were three of 83 cases on haloperidol. It is very difficult to arrive at definitive evidence about TD because most patients have received previous neuroleptic therapy and because TD-like symptoms occur spontaneously, providing an alternative explanation. It is clear that it is difficult to prove that olanzapine causes TD but equally difficult to prove that it does not. The 11-fold decreased incidence, however, is strong evidence that at least it produces much less TD. 

Stampfer MJ, Willett WC, Colditz GA, Speizer FE, Hennekens CH. Past use of oral contraceptives and cardiovascular disease a meta-analysis in the context of the Nurses Health Study. Am J Obstet Gynecol 1990 163(1 Pt 2) 285-91. 

From a clinical practice perspective, a clinician s treatment of a patient may be influenced by the results of a meta-analysis. Therefore, if the result is influenced by the fact that the articles included in the analysis were not truly representative of all evaluations of the treatment, the result is not likely to be representative either. The issue of publication bias therefore is of critical importance in the context of evidence-based medicine. 

From the position of drug development, the general requirement that scientific results have to be repeatable has been interpreted in the past by the Food and Drug Administration (the regulatory agency in the USA) to mean that two well-controlled studies are required to support a claim. But this requirement is itself controversial and its relation to a meta-analysis in the context of drug development is unclear. 

Chalmers I, Altman DG. Meta-analysis in context. In Systematic Reviews in Healthcare. London BMJ Publishing, 1995. 

This is an issue which has received much attention elsewhere but which is generally seen to be less relevant within the context of drug development. If one performs regular meta-analyses of clinical trials and then stops doing clinical trials if or when the result of the meta-analysis becomes signlflcant, then such a procedure is subject to the general bias to which (unadjusted) sequential clinical analysis of ordinary clinical trials is liable. 

Some argue that meta-analysis is meaningful only when researchers explicitly plan the study such that a prospective meta-analysis is possible. Research carries burdens in addition to those encountered in clinical context, such as extra follow-up visits, investigations and discomforts. These burdens cannot be justified by potential benefits to participants, but only by their ability to increase the value of the knowledge to be gained. 

In this chapter, we discuss the use of meta-analysis of randomized trials conducted in the regulatory context for the evaluation of safety. The chapter discusses key design, methodological, and reporting issues. We also present two real-life examples that had regulatory consequences and display many of these issues. 

Perhaps the most important concept of meta-analysis in the regulatory context is prespecification. This concept for meta-analysis is more multifaceted than for the case of a single trial. For example, a meta-analysis can be conducted based on completed trials. In this case, it is likely that the results from the completed trials influence the objectives and design of the meta-analysis. If a trial has identified a safety issue for a drug, including such a trial in a meta-analysis to confirm the safety signal is problematic, since in this case the meta-analysis will not provide independent findings. 

Good outcome ascertainment is an important component to a meta-analysis in the regulatory context. In the best-case scenario, the meta-analysis is conceived and designed prior to or in conjunction to the design of the trials to be included in the meta-analysis. In this case, the safety outcome can be collected prospectively and actively and if necessary can be adjudicated. By prospectively and actively collecting fhe outcome, ascertainment procedures can be designed to reduce misclassification and bias between the treatment arms of fhe frials. 

Meta-analysis, when applied appropriately, provides a powerful tool to evaluate safety. However, in the regulatory context, careful attention to rigor, robustness, and transparency is essential when performing meta-analysis. 

When this happens it is unlikely that methodological problems or systematic biases can influence the results of the studies conducted in different contexts and different study designs. The studies included in this meta-analysis usually controlled for such items as geographical location and date of birth, however, other potential confounding factors such as maternal age, alcohol, and smoking that could lead to subsequent problems in outcome presentation were not consistently reported. 

Orlitzky et al. (2003) conclude that market forces generally do not penalize companies that are high in corporate social performance and thus, managers can afford to be socially responsible. Their results of the meta-analysis revealed a positive association between social/environmental performance (CSP) and corporate financial performance (CFP) across industries and across study contexts. If managers believe that CSP is a prerequisite for CFP, they may actively pursue CSP in the belief that the market will reward them for such efforts. The company s executives and top-leadership must be attentive to the perceptions of third parties, regardless of whether they are market analysts, public interest groups or the media (Orlitzky et al. 2003). 

---