Robustness meta-analysis

Within any meta-analysis some trials will be larger than others and because of the way the trials are combined, the larger trials, i.e. those that have higher precision, will tend to dominate. It is helpful therefore to assess the robustness of the overall conclusion by omitting maybe the largest study or studies to see if the result remains qualitatively the same. If it does then the result is robust. If it does not then the overall result is undermined as it is then giving a result that is driven by the largest trial or trials. 

Another meta-analysis of placebo-controlled trials in depression published between 1980 and 2000 showed an increase in the response rates in the placebo arms of trials with a variety of antidepressants (Walsh et al.y 2002). Responses to placebo increased significantly in recent years, as shown by the high positive correlation with the year of publication. The association between response rate and year of publication was more statistically robust for placebo than for active medication. The change in placebo response rate did not appear to be explained directly by changes in study characteristics such as patient age, placebo lead-in or minimum required Hamilton Rating Scale for Depression score. A potential explanation could be the changing awareness of patients and the fact that many patients in recent clinical trials had been exposed to several previous treatments and thus expected to improve (see Box 5.4). 

Meta-analysis, when applied appropriately, provides a powerful tool to evaluate safety. However, in the regulatory context, careful attention to rigor, robustness, and transparency is essential when performing meta-analysis. 

Evaluation of the robustness of the result obtained from the meta-analysis... 

Neurobehavioral studies, as a public health template for lead toxicity, have been carried out around the world and have been of two types retrospective or cross-sectional and longitudinal or prospective studies. The retrospective study has more limits than a prospective one, in that the role of lead as an etiological agent is easier to define in prospective studies. Typically, groups are studied in the latter with much better control for confounding factors that could lower or obliterate any lead—effect relationship. The former appeared much earlier and varied in their quality. However, about a dozen or so retrospective studies from Europe and the United States show a lead—neurobehavioral effect relationship. When analyzed through inclusion in a meta-analysis, the robustness of the lead—IQ effect is amplified. 

An empirical meta-analysis approach was applied in order to develop a quantitative review of the available data and their aggregation towards response laws (Sauvant, 1992). The definite strength of this approach lays in the robustness of the empirically derived response laws, with the limit that the derived laws depend on the physiological and nutritional conditions associated to the data used. Their application is defined by the range of validity of the meta-design. Outside this range of validity, the derived predictions have to be used with caution. 

---