Type I and II errors

The aim of any clinical trial is to have low risk of Type I and II errors and sufficient power to detect a difference between treatments, if it exists. Of the three factors in determining sample size, the power (probability of detecting a true difference) is arbitrarily chosen. The magnitude of the drug s effect can be estimated with more or less accuracy from previous experience with drugs of the same or similar action, and the variability of the measurements is often known from published experiments on the primary endpoint, with or without the drug. These data will, however, not be available for novel substances in a new class and frequently the sample size in the early phase of development has to be chosen on an arbitrary basis. 

The chance of Type I and II errors should be considered. Alternative explanations for the results should be carefully addressed. Studies that discuss results in relationship to chance findings, random errors, possible confounders, or sources of bias should be weighed more heavily than are studies that ignore or incompletely address those issues. 

The aim of any clinical trial is to have small Type I and II errors and sufficient power to detect a difference between treatments, if it exists. Of the three factors in determining sample size, the power is arbitrarily chosen. The... 

Table 21.1 Type I and II errors in statistical decision making...

The number of experimental runs or experimental units is to be determined beforehand. Its precise determination involves statistical computation that requires prestated probability of committing type I and II errors, the desired accuracy in detecting the difference between the means of the responses resulting from different treatments. Besides, other statistical parameters are also required. As a rule of thumb, the appropriate size for an experiment is between 8 and 60. For more elaborate evaluation, consult statistical handbooks (Box et al. 1978 Daniel 1976 Wadsworth 1990 Winer et al. 1991). 

Differences in the Responses of the Different Types of Models. The basic differences that exist in the heat and mass balances for the different types of models determine deviations of the responses of types I and II with respect to type III. In a previous work (1) a method was developed to predict these deviations but for conditions of no increase in the radial mean temperature of the reactor (T0 >> Tw). In this work,the method is generalized for any values of T0 and Tw and for any kinetic equation. The proposed method allows the estimation of the error in the radial mean conversions of models I and II with respect to models III. Its validity is verified by comparing the predicted deviations with those calculated from the numerical solution of the two-dimensional models. A similar comparison could have been made with the numerical solution of the one-dimensional models. 

The existence of a region where fractions of errors type I and II are simultaneously small indicates that for some data series this test is able to clearly identify a stationary from a non-stationary point. However, for the case with several simultaneous variables that is analyzed later this is not necessarily true. 

It is also appropriate to note that not all clinical trials utilize formal sample size estimation methods. In many instances (for example, FTIH studies) the sample size is determined on the basis of logistical constraints and the size of the study thought to be necessary to gather sufficient evidence (for example, pharmacokinetic profiles) to rule out unwanted effects. However, when the objective of the clinical trial (for example, a superiority trial) is to claim that a true treatment effect exists while at the same time limiting the probability of committing type I or II errors (a and P), there are computational methods used to estimate the required sample size. The use of formal sample size estimation is required in therapeutic confirmatory trials, this book s major focus, and strongly suggested in therapeutic exploratory trials. 

The two error types mentioned in the title are also designated with the Roman numerals I and II the associated error probabilities are termed alpha (a) and beta ( 8). 

Another important aspect is to ensure that we limit the errors in drawing the wrong conclusion. These are described as Type I and Type II errors ... 

Once the parameters for the hypothesis and Type I and Type II errors are set, the total number of subjects (IN) to be recruited to join the trial can be determined by the equation... 

Many people ask at this stage why 0.05 Well it is in one sense an arbitrary choice, the cut-off could easily have been 0.04 or 0.075 but 0.05 has become the agreed value, the convention. We will explore the implications of this choice later when we look at type I and type II errors. 

The statistical test procedures that we use unfortunately are not perfect and from time to time we will be fooled by the data and draw incorrect conclusions. For example, we know that 17 heads and 3 tails can (and will) occur with 20 flips of a fair coin (the probability from Chapter 3 is 0.0011) however, that outcome would give a significant p-value and we would conclude incorrectly that the coin was not fair. Conversely we could construct a coin that was biased 60 per cent/40 per cent in favour of heads and in 20 flips see say 13 heads and 7 tails. That outcome would lead to a non-significant p-value (p = 0.224) and we would fail to pick up the bias. These two potential mistakes are termed type I and type II errors. 

CH08 POWER AND SAMPLE SIZE Table 8.1 Type I and type II errors... 

Although conventional p-values have no role to play in equivalence or noninferiority trials there is a p-value counterpart to the confidence intervals approach. The confidence interval methodology was developed by Westlake (1981) in the context of bioequivalence and Schuirmann (1987) developed a p-value approach that was mathematically connected to these confidence intervals, although much more difficult to understand It nonetheless provides a useful way of thinking, particularly when we come later to consider type I and type II errors in this context and also the sample size calculation. We will start by looking at equivalence and use A to denote the equivalence margins. 

On the chart are two boundary lines, the positions of which depend upon the risks a and p, of errors of type I and type II, the magnitude of the difference it is important to detect, etc. The lines divide the chart into three zones (1) in which the Null Hypothesis is accepted (2) in which the alternative hypothesis is accepted and (3) in which there is no decision. 

Obviously tests and decisions drawn from them are dependent on the estimated (calculated) parameters. It is worth mentioning that they also depend on the number, n, of measurements or replicates. In addition we know that we do not have one risk alone. Therefore we have to accept that the sensitivity of tests is determined by the proper choice of n, it increases with increasing n. It is, however, not usually possible to increase n without additional costs. On the other hand one can often say, e.g. from experience or from legal requirement which difference between two means may be tolerated and which difference must be assured. Taking into account this and both tolerable risks of wrong decisions, associated with type I and type II errors, one can compute the necessary number of replicates in advance (see textbooks on statistics or ZWANZIGER et al. [1986]) ... 

Table 3. THG measurements of the second-order hyperpolarizabilities yfor substituted Type I and Type II hydrazone derivatives in the solvent DMF at a fundamental wavelength of X =1907 nm. The aromatic ring Ar is either benzene or thiophene (see Fig. 20). is the extinction coefficient at the harmonic wavelength of the THG experiment (reference = 1.6x10 22 m2/V2,10% experimental error)...

---