Sample Size Issues

TABLE 10.5 Lymphoma Data Results Using Bootstrap Approaches for Model Selection and Performance Assessment 

Bootstrap Prediction error Sensitivity Specificity PPV NPV A -Value 

We utilized both the leave-one-out bootstrap as well as the 0.632+ alternative to estimate the error rate of k-NN in the lymphoma dataset. In Table 10.5, LOOB s overestimation of the error rate is quite obvious compared to the previous methods. The 0.632+ estimator correction is also apparent. Both have a significantly lower value for k compared with other methods. A k-value of 1 indicates that the best method of predicting the l5miphoma subtype for a given observation is to look at the subt5q)e of the one closest observation in terms of the genetic expression values. 

Thus far, as seen in the lymphoma dataset example, the difference in prediction error estimation between methods is trivial. In part, this is due to the large observed sample size of n = 240. The effect of sample size on estimation and model selection becomes evident when we restrict the sample to a smaller number of patients. As before, for the 

Each random sample plays two roles. First, it serves as a sample fi om which to calculate each resampling method s estimate of prediction error, as though we only observed 20 patients total. Second, it serves as a training set while the remaining 220 observations serve as the test set for an approximation of the true error. For feature selection, we used univariate t-statistics to identify the 10 most differentially expressed genes and -NN to build the rule based on these 10 genes. 

---

A helpful tutorial on sample size issues is the paper by Steven Julious in Statistics in Medicine (Julious, 2004) a classic text is that of Desu and Raghavarao (1990). Nowadays, the use of specialist software for sample size determination such as NQuery, PASS or Power and Precision is common. 

How many products will be collected (statistical sample size), i.e., will the tail of the distribution need to be defined, or will the mean sufficiently address the issue of concern ... 

Growing experience with complex disease genetics has made clear the need to minimize type I error in genetic studies [41, 109]. Power is especially an issue for SNP-based association studies of susceptibility loci for phenomenon such as response to pharmacological therapy, which are extremely heterogeneous and which are likely to involve genes of small individual effect. Table 10.2 shows some simple estimation of required sample sizes of cases needed to detect a true odds ratio (OR) of 1.5 with 80% power and type I error probability (a) of either 0.05 or 0.005. 

Sample size and matrix Your choice of analytical method will also be dependent on the amount of sample you have, especially if the amount is limited and some of the methods under consideration are destructive to the sample. In the Bulging Drum Problem, sample size was not an issue. However, sampling the gas in the drum was challenging, since loss and contamination were quite likely. Getting the samples to the lab presented other challenges. Sample matrix is another important factor in method choice. As you know, some methods and instrumental techniques are not suitable for analysis of solids, without sample preparation. Table 21.8 lists some of the issues that must be considered for different sample matrices. 

Several specialized reviews on detection of QT liability in the clinical development phase have already been published and the reader is referred to these publications [63]. Guidelines of the International Society for Holter and Noninvasive Electrocardiology (IS H N E) for electrocardiographic evaluation of drug-related QT prolongation are also available [161]. The main issues related to measurement of the QT interval in clinical studies are summarized in Table 3.4. An important aspect is the calculation of sample size usually 40-60 subjects per treatment arm are required, implying high cost [162-164]. 

There are of course practical considerations in clinical research. We may find patient recruitment difficult in single centre studies and this is one of the major drivers to multicentre and multinational trials. Alternatively, we may need to relax the inclusion/exclusion criteria or lengthen the recruitment period. Unfortunately, while each of these may indeed increase the supply of patients they may also lead to increased variability that in turn will require more patients. A second issue is the size of the CRD which, if it is too small, will require a large number of patients. In such circumstances we may need to consider the use of surrogate endpoints (Section S.3.3.2). Finally, the standard deviation may be large and this can have a considerable impact on the sample size - for example, a doubling of the standard deviation leads to a four times increase in the... 

Also, the analysis plan should identify the statistical methods that will be used and how hypotheses will be tested (e.g., a p value cutoff or a confidence interval for the difference that excludes 0). And the plan should prespecify whether interim analyses are planned, indicate how issues of multiple testing will be addressed, and predefine any subgroup analyses that will be conducted. Finally, the analysis plan should include the results of power and sample size calculations. 

As with binary and categorical data, is there an issue with small sample sizes Well, in fact, no, there is not. The MH test is a different kind of chi-square test and is not built around expected frequencies. As a consequence it is not affected by small expected frequencies and can be used in all cases for ordinal data. There are some pathological cases where it will break down but these should not concern us in practical settings. 

We commonly refer to the level of effect to be detected as the cliniMlly relevant difference (crd) what level of effect is an important effect from a clinical standpoint. Note also that crd stands for commercially relevant difference it could well be that the decision is based on commercial interests. Finally crd stands for cynically relevant difference It does happen from time to time that a statistician is asked to do a sample size calculation, oh and by the way, we want 200 patients The issue here of course is budget and the question really is what level of effect are we able to detect with a sample size of 200 ... 

This document has set down some initial thoughts from a regulatory point of view about the issues involved in allowing the design of a clinical trial to be adapted as the trial progresses. Modification of the sample size based on blinded data and stopping for overwhelming efficacy or futility are forms of adaptation that are already well accepted, but this Reflection Paper considers other possibilities that are more controversial. 

Quantity of sample is also an issue. The choice of sample mass should be such that signal-to-noise ratio is optimum. If the mass loss is very small, use of larger samples is helpful. However, if the sample size is too large, thermal... 

It is important to distinguish between acute (e.g., 1 day) and longer term treatment with a BZD. Most of the literature addressing this issue consists of anecdotal reports, retrospective chart reviews, and uncontrolled studies in small patient samples, plus a small number of controlled trials for short-term acute BZD therapy. To our knowledge, at least 12 studies (including more than 450 patients) have evaluated the efficacy of adjunctive BZDs in nonresponsive schizophrenics (Table 5-23). Two of three open studies showed positive results, as did two controlled, single-blind studies ( 343, 344, 345, 346 and 347). In seven double-blind, crossover studies (six with placebo controls), the results are more contradictory, in that five showed no advantage to an adjunctive BZD, and one of the two positive studies had a small sample size (188, 189, 348, 349, 350 and 351). 

---