Fixed-sample size trials

An alternative (or addition) to repeating the fixed-sample size trial is to use a sequential design in which the trial is run until a useful result is reached. These adaptive designs, in which decisions are taken on the basis of results to date, can assess results on a continuous basis as data for each subject that becomes available or, more commonly, on groups of subjects... 

As Whitehead (1997) points out, this can even occur with fixed sample size studies of survival analysis. For example, consider a fixed sample size trial with a recruitment over one year. We may have determined to analyse the results once the last patient recruited has been followed up for one year. This means, though, that patients recruited earlier in the trial will have been followed up longer those recruited at the beginning will, in fact, have been followed up for two years. Often this extra information will also be included in the analysis, which thus reflects a mixture of follow-up times from one to two years. However, once this analysis is complete it will still be possible to obtain further data and in a year s time an analysis of patients with 2-3 years follow up could be carried out. I think it is fair to say, however, that it is more likely to be a problem which makes itself known in a sequential rather than a fixed trial framework. Nevertheless, it is a potential feature of all forecasting systems that they are hostages to the future further information which embarrasses us can always arise. 

This feature seems to be embarrassing for frequentist systems where there seems to be no formal way of declaring a hypothesis not rejected once it has been rejected. It does not appear to be a reversible process. However, this is not the only interpretation of what goes on in a sequential analysis with over-running. One view is that the only analysis which counts is the final one, once the further information has been collected. The trial has been stopped earlier on the practical grounds that it seems as if a decision of efficacy is now possible. There is no question of unrejecting Hq. It was not rejected in the first place. After all, in a fixed sample size trial, we would have made a decision when to stop the trial and then some time later we would either have found that the result would or would not have been significant. The only difference with the sequential trial is that the decision when to stop has made some use of the results. 

Table 1. Study differences detectable given a fixed sample size. Values represent minimum detectable differences between trial arms given the standard deviation reported for the row in the table, and a fixed sample size for each arm of the trial...

In the fixed sample clinical trial approach, one analysis is performed once all of the data have been collected. The chosen nominal significance level (the Type I error rate) will have been stated in the study protocol and/or the statistical analysis plan. This value is likely to be 0.05 As we have seen, declaring a finding statistically significant is typically done at the 5% p-level. In a group sequential clinical trial, the plan is to conduct at least one interim analysis and possibly several of them. This procedure will also be discussed in the trial s study protocol and/or the statistical analysis plan. For example, suppose the plan is to perform a maximum of five analyses (the fifth would have been the only analysis conducted had the trial adopted a fixed sample approach), and it is planned to enroll 1,000 subjects in the trial. The first interim analysis would be conducted after data had been collected for the first fifth of the total sample size, i.e., after 200 subjects. If this analysis provided compelling evidence to terminate the trial, it would be terminated at that point. If compelling evidence to terminate the trial was not obtained, the trial would proceed to the point where two-fifths of the total sample size had been recruited, at which point the second interim analysis would be conducted. All of the accumulated data collected to this point, i.e., the data from all 400 subjects, would be used in this analysis. 

Fixed-sample size and sequential designs SENSITIVITY OF TRIALS... 

Defining when a clinical trial should end is not as simple as it first appears. In the standard clinical trial the end is defined by the passage of all of the recruited subjects through the complete design. But, it is results and decisions based on the results that matter, not the number of subjects. The result of the trial may be that one treatment is superior to another or that there is no difference. These trials are of fixed-sample size. In fact, patients are recruited sequentially, but the results are analysed at a fixed time-point. The results of this type of trial may be disappointing if they miss the agreed and accepted level of significance. 

So, if there is a place for sequential analysis of drug-development trials, monitoring and so forth, it is either for safety and/or lack of efficacy reasons, or for economic reasons. The latter can be important for some indications. There may be cases where the sponsor realizes that the number of patients needed to prove efficacy may, for a fixed trial, exceed those which need to be studied for registration. Under such circumstance, the possibility of early stopping, either for proven efficacy or lack of efficacy (what was referred to in section 19.1 as futility), may be attractive. The expected run length will be less than the fixed sample size required, the costs of the trial may be less, and the time to registration if the trial is successful will probably be reduced. 

To obtain the empirical estimates of a, Kowalski and Hutmacher (33) simulated 300 chnical trials for each combination of sample size and p, where the proportional reduction in CUP (0) was fixed to zero. Covariate and base models were fitted to each of the trials and the likelihood ratio tests were performed at the 5% level of significance. The percentage of trials where a statistically significant difference in CUP was observed provided an empirical estimate of a (i.e, PIoi = 0 is rejected when i/o is true). The data were analyzed with the NONMEM population phar-macokinetics/pharmacodynamics analysis software. The results suggested that an approximate nine-point change in the objective function should be used to assess statistical significance at the 5% level rather than the commonly used critical value of 3.84 for one degree of freedom. 

The implications of the disparity between empirical and nominal significance levels of the likelihood ratio test in mixed effects modeling and simulation are clear however, definitive solutions or corrections are not. While the significance of random effects is not generally the subject of interest in a simulation, the bias in hkelihood ratio test-determined p value for fixed effects could be very influential on trial simulation findings. Thus, simulation exercises should provide for determination of empirical p values to avoid faulty conclusions about power and sample size. 

Now, from (13.7) for given prior odds and fixed a, the posterior odds are greater the smaller the value of (3, which is to say the greater the power of the test. But the power increases with sample size. Hence, other things being equal, significant results are more indicative of efficacy if obtained from large trials rather than small trials. 

The table is arranged in terms of the product, Np, where N is the sample size and p is the fixed probability for the entire population. For this distribution Np = p. = c, i.e., both the mean and the variance are equal to Np. The standard deviation is a =. yi. Values in the body of the table represent the cumulative probability of X or more successes in N trials (the same as for the binomial table) or in sampling, the values represent the probability of X or more acceptances in sample of N items. In either case the fixed probabihty for the whole population is p. 

---