Significance level interim analysis

Timing of any interim analysis Reason for sample size Power of clinical study Level of significance Criteria for termination of study Procedure for missing and imused data Reporting procedures for deviations from statistical plan... 

Pocock (1977) developed a procedure which divides the type I error rate of 5 per cent equally across the various analyses. In the example above with two interim looks and a final analysis, Bonferroni would suggest using an adjusted significance level of 0.017 (= 0.05 4- 3). The Pocock method however gives us the correct adjusted significance level as 0.022 and this exactly preserves the overall 5 per cent type I error rate. 

One area that we briefly mentioned was interim analysis, where we are looking at the data in the trial as it accumulates. The method due to Pocock (1977) was discussed to control the type I error rate across the series of interim looks. The Pocock methodology divided up the 5 per cent type I error rate equally across the analyses. So, for example, for two interim looks and a final analysis, the significance level at each analysis is 0.022. For the O Brien and Fleming (1979) method most of the 5 per cent is left over for the final analysis, while the first analysis is at a very stringent level and the adjusted significance levels are 0.00052, 0.014 and 0.045. 

These two methods are the most common approaches seen in pharmaceutical applications. A third method, which we see used from time to time, is due to Haybittle (1971) and Peto et al. (1976). Here a significance level of 0.001 is used for each of the interims, again leaving most of the 5 per cent left over for the final analysis. For two interims and a final, the adjusted significance level for the final analysis is in fact 0.05 to two decimal places, for three interims we have 0.049 left over . Clearly these methods have little effect on the final evaluation, but associated with that there is also little chance of stopping at an interim stage. 

In line with this, the interim analysis plan was revised and only one interim was to be conducted for both efficacy and futility after 40 per cent of the patients had completed 3 months of follow-up. Since the two proposed analyses were not equally spaced, a spending functions were needed to revise the adjusted significance levels and these turned out to be 0.0007 and 0.0497. 

In the fixed sample clinical trial approach, one analysis is performed once all of the data have been collected. The chosen nominal significance level (the Type I error rate) will have been stated in the study protocol and/or the statistical analysis plan. This value is likely to be 0.05 As we have seen, declaring a finding statistically significant is typically done at the 5% p-level. In a group sequential clinical trial, the plan is to conduct at least one interim analysis and possibly several of them. This procedure will also be discussed in the trial s study protocol and/or the statistical analysis plan. For example, suppose the plan is to perform a maximum of five analyses (the fifth would have been the only analysis conducted had the trial adopted a fixed sample approach), and it is planned to enroll 1,000 subjects in the trial. The first interim analysis would be conducted after data had been collected for the first fifth of the total sample size, i.e., after 200 subjects. If this analysis provided compelling evidence to terminate the trial, it would be terminated at that point. If compelling evidence to terminate the trial was not obtained, the trial would proceed to the point where two-fifths of the total sample size had been recruited, at which point the second interim analysis would be conducted. All of the accumulated data collected to this point, i.e., the data from all 400 subjects, would be used in this analysis. 

In the context of such interim analysis—repeated analysis to see if the drug response, on average, differs from the placebo response—the same hypothesis is tested five times if the trial is not terminated early and proceeds to its final stage. While the chance of erroneously obtaining a significant result at the 5% level of significance is 1 in 20 if only one analysis is conducted, this chance increases every time more than one test is performed. 

The FDA s statisticians expect that each clinical protocol submitted will include a description of any planned interim analyses of the data, including the anticipated monitoring procedures, the variables to be analyzed, and the statistical methods to be applied. The latter includes the choice of significance level for each interim analysis, the frequency of analysis, and a declaration of the circumstances under which the study would be terminated before the planned number of patients have been entered into the study. 

A group sequential approach was planned to discharge the 1.3 threshold. The significance levels for the multiple analyses were based on the Lan-DeMets spending function with an O Brien-Fleming boundary. The first analysis to discharge it would be conducted at the time when the meta-analysis to discharge the 1.8 threshold was performed. The next planned interim meta-analysis would be conducted when approximately 500 MACE-plus events had accrued, and a final interim analysis was planned to occur after approximately 700 events had accumulated, if the E3... 

---