Clinical trials error types

The aim of any clinical trial is to have low risk of Type I and II errors and sufficient power to detect a difference between treatments, if it exists. Of the three factors in determining sample size, the power (probability of detecting a true difference) is arbitrarily chosen. The magnitude of the drug s effect can be estimated with more or less accuracy from previous experience with drugs of the same or similar action, and the variability of the measurements is often known from published experiments on the primary endpoint, with or without the drug. These data will, however, not be available for novel substances in a new class and frequently the sample size in the early phase of development has to be chosen on an arbitrary basis. 

Sample sizes for clinical trials are discussed more fully elsewhere in this book and should be established in discussion with a statistician. Sample sizes should, however, be sufficient to be 90% certain of detecting a statistically significant difference between treatments, based on a set of predetermined primary variables. This means that trials utilising an active control will generally be considerably larger than placebo-controlled studies, in order to exclude a Type II statistical error (i.e. the failure to demonstrate a difference where one exists). Thus, in areas where a substantial safety database is required, for example, hypertension, it may be appropriate to have in the programme a preponderance of studies using a positive control. 

When multiplicity is present, the usual frequentist approach to the analysis of clinical trial data may necessitate an adjustment to the type I error. Multiplicity may arise for, example. 

Table 9.1 shows that the results from a clinical trial can lead the sponsor to an inappropriate conclusion in some cases. In these cases, one of two types of error occurs ... 

In the fixed sample clinical trial approach, one analysis is performed once all of the data have been collected. The chosen nominal significance level (the Type I error rate) will have been stated in the study protocol and/or the statistical analysis plan. This value is likely to be 0.05 As we have seen, declaring a finding statistically significant is typically done at the 5% p-level. In a group sequential clinical trial, the plan is to conduct at least one interim analysis and possibly several of them. This procedure will also be discussed in the trial s study protocol and/or the statistical analysis plan. For example, suppose the plan is to perform a maximum of five analyses (the fifth would have been the only analysis conducted had the trial adopted a fixed sample approach), and it is planned to enroll 1,000 subjects in the trial. The first interim analysis would be conducted after data had been collected for the first fifth of the total sample size, i.e., after 200 subjects. If this analysis provided compelling evidence to terminate the trial, it would be terminated at that point. If compelling evidence to terminate the trial was not obtained, the trial would proceed to the point where two-fifths of the total sample size had been recruited, at which point the second interim analysis would be conducted. All of the accumulated data collected to this point, i.e., the data from all 400 subjects, would be used in this analysis. 

By its nature, therefore, the group sequential design involves the possibility of multiple testing. In this example it is possible that five analyses could be conducted on data collected in this clinical trial. As discussed in Section 7.10, there is an inherent problem with multiple testing. As more tests are performed, it becomes increasingly likely that a Type I error will occur, i.e., that a result will erroneously be declared as statistically significant. As also noted at that point, however, the problem can be addressed completely satisfactorily by taking appropriate statistical care. 

The pharmaceutical physician may not be expected to be a specialist statistician, and statistics are not the subject of this chapter. However, the ability to talk to and understand statisticians is absolutely essential. Sine qua non Involve a good statistician from the moment a clinical trial is contemplated. Furthermore, the pharmaceutical physician should be confident of a sound understanding of the concepts of type I and type II error, and the probabilities a and P (e.g. Freiman et al., 1978). This is one of your best defences against bias. 

The aim of any clinical trial is to have small Type I and II errors and sufficient power to detect a difference between treatments, if it exists. Of the three factors in determining sample size, the power is arbitrarily chosen. The... 

It is also appropriate to note that not all clinical trials utilize formal sample size estimation methods. In many instances (for example, FTIH studies) the sample size is determined on the basis of logistical constraints and the size of the study thought to be necessary to gather sufficient evidence (for example, pharmacokinetic profiles) to rule out unwanted effects. However, when the objective of the clinical trial (for example, a superiority trial) is to claim that a true treatment effect exists while at the same time limiting the probability of committing type I or II errors (a and P), there are computational methods used to estimate the required sample size. The use of formal sample size estimation is required in therapeutic confirmatory trials, this book s major focus, and strongly suggested in therapeutic exploratory trials. 

Statistics plays a major role in the design of the clinical trial. The groups or subgroups to be studied, the frequencies, dosages, and the markers to monitor drug efficacy are all important factors to consider. The statistical analysis provides the tool to demonstrate, at a certain confidence level, whether the drug is effective. This is normally reported in the form of a statistical power test, analyzing the Type I and Type II errors. 

Gould AL (1992) Interim Analyses for monitoring clinical trials that do not materially affect the type-I error rate. Statistics in Medicine 11 55-66. 

If we now look at the ability of various types of clinical trial to resolve total variation, then the situation is as illustrated in Table 25.5, which is also taken from Senn (2001). We can see that, as discussed above, a single cross-over trial is not capable of distinguishing between sources of variation C and D. This is the situation discussed in Section 25.2.1. A slight caveat must be entered here. If more than two treatments are being studied in a cross-over trial, then some partial identification of treatment-by-patient interaction is possible. This is because in a cross-over trial in n and t treatments in t periods the nt —1 degrees of freedom resolve into n — for patients and t— for each of treatments and periods. This leaves nt — n — lt + liot error. If r = 2 the degrees of freedom reduce ton-2, which is fewer than the number of patients. However, if r > 3 then the residual degrees of freedom for even moderately sized trials exceed the number of patients and this means that some partial identification is possible (Senn, 2002 Senn and Hildebrand, 1991). Nevertheless, full identification requires full replication of patient-treatment combinations. 

Size (of a test). When used in connection with hypothesis testing, the size of a test is the probability of rejecting the null hypothesis given that the null hypothesis is true. (In other words, it is the probability of committing a type I error when the null hypothesis is true.) Conventionally in clinical trials a size of 5% is used. Also called significance level. 

One hundred twenty-five events would provide 90% power, and 95 events would provide 80% power to illustrate the 95% Cl is < 1.8 based on a Cox proportional hazard model. Therefore, in a clinical trial with an expected event rate of 1 event per 100 patient-years and a 5% annual dropout rate, 90% power would be achieved by enrolling 200 patients per month for 24 months and following all enrolled patients for an additional 24 months. The total trial duration would be 48 months. If, however, the true event rate was actually 0.75%, the power would decrease to 80%—a doubling of the type 2 error. Furthermore, if the event rate were actually 0.5%, then the power would drop to 65%, a type 2 error 3.5 times as high. Because it is hard to predict CV event rates in noncardiac populations, such overestimates are not uncommon. Similarly, if the actual event rate... 

---