Statistical analysis plan multiplicity

From a statistical point of view, compelling evidence of unexpected adverse events is the hardest to address satisfactorily. When unanticipated safety concerns arise, the fact that they are unanticipated means by definition that they would not have been addressed in the study protocol or statistical analysis plan and that no prespecified analytical strategy is in place. Additionally, file vast range of possible adverse events that might be anticipated means that controlling adequately for multiplicity problems is difficult (Ellenberg et al., 2003). 

SMQs) might be an interesting future direction as the SMQs represent more well-established medical concepts. Because there is flexibility in modeling different levels of AEs, as a practical advice, it is important to prespecify the multiplicity adjustment method that will be used. Grouping and definitions should be described in the study protocol or the statistical analysis plan, rather than on a post hoc basis. 

Also, the analysis plan should identify the statistical methods that will be used and how hypotheses will be tested (e.g., a p value cutoff or a confidence interval for the difference that excludes 0). And the plan should prespecify whether interim analyses are planned, indicate how issues of multiple testing will be addressed, and predefine any subgroup analyses that will be conducted. Finally, the analysis plan should include the results of power and sample size calculations. 

The data were analysed in several stages. Descriptive statistics and bivariate correlations were calculated for independent, dependent and control variables. Control variables with significant bivariate correlations with outcome as measured by the NBAS scales were used in forward stepwise multiple regression analyses to determine the best joint predictors of the NBAS. After the best multiple regression model was constructed from the non-lead variables, lead measurements recorded at the three time points were added to the model to determine the relationship between each of the lead measurements and the adjusted NBAS scores. The overall plan of analysis follows Bellinger et al (1984 this volume). Analyses were performed with SAS programs. 

The nature of the test batteries, comprising a large number of tests, has demonstrated the problems of multiple non-independent outcomes. In some cases, the analyses have been carried out as if the tests were all independent, and with a large number of exploratory analyses in a search for significant results. Since significance is determined on the basis of probabilities these fishing expeditions, as they have been called, are bound to produce some spurious results. It is now clear that it is essential to have a predetermined analysis policy and plan, and for analyses to be carried out in accordance with this. In addition, the potential confounders should be defined on the basis of prior hypothesis of their association with the outcome, and not only because of their identified statistical association. This will have the effect of reducing bias in multivariate analysis with confounders. 

---