The 95 per cent confidence interval

We have seen in the previous chapter that it is not possible to make a precise statement about the exact value of a population parameter, based on sample data, and that this is a consequence of the inherent sampling variation in the sampling process. The confidence interval provides us with a compromise rather than trying to pin down precisely the value of the mean p or the difference between two means — p2 for example, we give a range of values, within which we are fairly certain that the true value lies. 

We will first look at the way we calculate the confidence interval for a single mean p and then talk about its interpretation. Later in this chapter we will extend the methodology to deal with pj — p2 and other parameters of interest. 

In the computer simulation in Chapter 2, the first sample ( = 50) gave data (to 2 decimal places) as follows  

The lower end of the confidence interval, the lower confidence limit, is then given by  

---

The second sample in the computer simulation gave the following data x = 80.77 mmHg and s = 4.50mmHg and this results in the 95 per cent confidence interval as (79.52, 82.02). Again we see that the interval has captured the true mean. 

Now look at all 100 samples taken from the normal population with p = 80 mmHg. Figure 3.1 shows the 95 per cent confidence intervals plotted for each of the 100 simulations. A horizontal line has also been placed at 80 mmHg to allow the confidence intervals to be judged in terms of capturing the true mean. 

Most of the 95 per cent confidence intervals do contain the true mean of 80 mmHg, but not all. Sample number 4 gave a mean value 3c = 81.58 mmHg with a 95 per cent confidence interval (80.33, 82.83), which has missed the true mean at the lower end. Similarly samples 35, 46, 66, 98 and 99 have given confidence intervals that do not contain p = 80 mmHg. So we have a method that seems to work most of the time, but not all of the time. For this simulation we have a 94 per cent (94/100) success rate. If we were to extend the simulation and take many thousands of samples from this population, constructing 95 per cent confidence intervals each time, we would in fact see a success rate of 95 per cent exactly 95 per cent of those intervals would contain the true (population) mean value. This provides us with the interpretation of a 95 per cent confidence interval in... 

Just as an aside, look back at the formula for the 95 per cent confidence interval. Where does the 1.96 come from It comes from the normal distribution 1.96 is the number of standard deviations you need to move out to, to capture 95 per cent of the values in the population. The reason we get the so-called 95 per cent coverage for the confidence interval is directly linked to this property of the normal distribution. 

The formula for the 95 per cent confidence interval (and also for the 99 per cent confidence interval) given above is in fact not quite correct. It is correct up to a... 

For sample sizes beyond about 30 the multiplying constant for the 95 per cent confidence interval is approximately equal to two. Sometimes for reasonably large sample sizes we may not agonise over the value of the multiplying constant and simply use the value two as a good approximation. This gives us an approximate formula for the 95 per cent confidence interval as (3c — 2se, x + 2se). 

At the end of the previous chapter we saw how to extend the idea of a standard error for a single mean to a standard error for the difference between two means. The extension of the confidence interval is similarly straightforward. Consider the placebo controlled trial in cholesterol lowering described in Example 2.3 in Chapter 2. We had an observed difference in the sample means 3cj — 3c2 of 1.4 mmol/1 and a standard error of 0.29. The formula for the 95 per cent confidence interval for the difference between two means — p.2) i -... 

For the trastuzumab group the 95 per cent confidence interval for Oj is given by ... 

Having calculated the p-value we would also calculate the 95 per cent confidence interval for the difference — P2 to give us information about the magnitude of the treatment effect. For the data in the example in Section 3.3.3 this confidence interval is given by ... 

The 95 per cent confidence interval a, b) for the difference in the treatment means, — p2> provides a range of plausible values for the true treatment difference. With 95 per cent confidence we can say that pj — p2 is somewhere within... 

This link applies also to the p-value from the unpaired t-test and the confidence interval for p, the mean difference between the treatments, and in addition extends to adjusted analyses including ANOVA and ANCOVA and similarly for regression. For example, if the test for the slope b of the regression line gives a significant p-value (at the 5 per cent level) then the 95 per cent confidence interval for the slope will not contain zero and vice versa. 

It is all too common to see a conclusion that treatments are the same (or similar) simply on the back of a large p-value this is not necessarily the correct conclusion. Presentation of the 95 per cent confidence interval will provide a statement about the possible magnitude of the treatment difference. This can be inspected and only then can a conclusion of similarity be made if this interval is seen to exclude clinically important differences. We will return to a more formal approach to this in Chapter 12 where we will discuss equivalence and non-inferiority. 

The next step is to undertake the trial and calculate the 95 per cent confidence interval for the difference in the means (mean increase in PEF on new inhaler (pi) - mean increase in PEF on existing inhaler ( 2))- As a first example, suppose that this confidence interval is (-71/min, 121/min). In other words, we can be 95 per cent confident that the true difference, pj — P2, is between 71/min in favour of the existing inhaler and 121/min in favour of the new inhaler. 

In contrast, suppose that the 95 per cent confidence interval had turned out to be (—171/min, 121/min). This interval is not entirely within the equivalence margins and the data are supporting potential treatment differences below the lower equivalence margin. In this case we have not established equivalence. 

Step 2 is then to run the trial and compute the 95 per cent confidence interval for the difference, Pi — P2> in the mean reductions in diastolic blood pressure. In the above example suppose that this 95 per cent confidence interval turns out to be ( — 1.5 mmHg, 1.8 mmHg). As seen in Figure 12.2, all of the values within this interval are compatible with our definition of non-inferiority the non-inferiority of the test treatment has been established. In contrast, had the 95 per cent confidence interval been, say, (—2.3 mmHg,... 

In a clinical trial with the objective of demonstrating non-inferiority, suppose that the data are somewhat stronger than this and the 95 per cent confidence interval is not only entirely to the right of — A, but also completely to the right of zero as in Figure 12.5 there is evidence that the new treatment is in fact superior. 

If the 95 per cent confidence interval for the treatment effect not only lies entirely above - A but abo above zero, then there is evidence of superiority in terms of statistical significance at the 5 per cent level (p < 0.05). In this case, it is acceptable to calculate the exact probability associated with a test of superiority and to evaluate whether this is sufficiently small to reject convincingly the hypothesis of no differenc... Usually this demonstration of a benefit is sufficient for licensing on its own, provided the safety profiles of the new agent and the comparator are similar. ... 

Following calculation of the exact p-value for superiority the 95 per cent confidence interval allows the clinical relevance of the finding to be evaluated. Presumably, however, any level of benefit would be of value given that at the outset we were looking only to demonstrate non-inferiority. 

The 95 per cent confidence interval for 0 — 0p was in fact (7 per cent, 31 per cent) and this is entirely to the right, not only of —15 per cent, but also of zero. In this case a claim for the superiority of fluconazole is supported by the data. The authors concluded that non-inferiority had been established, but additionally there was evidence that fluconazole was more effective than amphotericin B ... 

For the imipramine tablets, Table 5.2 shows confidence limits of 24.70 and 25.94 mg. Therefore, we can state, with 95 per cent confidence, that if we returned to this batch of tablets and took larger and larger samples, the mean imipramine content would eventually settle down to some figure no less than 24.70 mg and no greater than 25.94 mg. This can be conveniently presented visually as in Figure 5.3. The dot indicates the point estimate and the horizontal bar represents the extent of the 95 per cent confidence interval. 

Figure 5.4 shows simulated samples of nine tablets taken from a large batch, for which the true mean imipramine content is 25.0 1.0 mg ( SD). Each horizontal bar represents the 95 per cent confidence interval from one of the samples. Out of the 30 samples, we would expect 95 per cent to produce intervals that include the true population mean and the remainder (one or two cases) will be unusually misleading... 

Figure 5.5 The 95 per cent confidence intervals for mean imipramine content. Population mean = 25 mg. Sample size = 9. The SD varies between 0.2 and 2 mg...

Figure 5.12 The 95 per cent confidence interval for mean pesticide content calculated (a) directly or (b) via Log transformation...

Figure 6.4 The 95 per cent confidence interval for the difference in theophylline clearance between control and rifampicin treated subjects...

Figure 6.5 made the point that we are looking to see if the 95 per cent confidence interval crosses (and therefore includes) the zero line. Two things influence the likelihood of a significant outcome and these are shown in Figures 6.7 and 6.8. 

Figure 6.7 shows the influence of the size of the experimental effect. If the mean clearances differ to only a very small extent (as in the two lower cases in Figure 6.7), then the 95 per cent confidence interval will probably overlap zero, bringing a non-significant result. However, with a large effect (as in the two upper cases), the confidence interval is displaced well away from zero and will not overlap it. 

One of the factors that feeds into the calculation of a two-sample /-test is the sample size. If we investigate a case where there is a real difference, but use too small a sample size, this may widen the 95 per cent confidence interval to the point where it overlaps zero. In that case, the results would be declared non-significant. This is a different kind of error. We are now failing to detect an effect that actually is present. This is a false negative or type II error . 

Standard of proof demanded (a) A formal calculation of power technically needs to take into account the standard of proof being required for a declaration of significance. The usual criterion is that the 95 per cent confidence interval excludes a zero effect (a =0.05). If an experiment was designed to achieve a higher standards of proof (e.g. a = 0.02), a 98 per cent Cl will have to be used and this will be wider than the standard 95 per cent CL The wider interval is then more likely to cross the zero line and so power will be lower. So, requiring a lower risk of a false positive (reducing alpha) will lead to less power. 

The 95 per cent confidence interval is 70.9-92.8 per cent, so the most we can say is that, if this therapy were implemented, we would expect the success rate to settle down eventually somewhere within the rather broad range of about 71-93 per cent. 

Figure 22.2 Gaussian curves. When the number of measurements increases and if the interval determining the group is narrow, then the graphical shape will take (measurement/frequency) the form of a Gaussian curve (Normal distrihution law). Below, two series of results centred on two different true means /t. If the numher of measurements is very small, it is not possible to estimate the average distribution, thus the true mean. At the bottom right, a reduced form of the Gaussian distribution is shown. The reliabUity of the mean is given by the 95 per cent confidence interval.

---