The unpaired t-test

In Section 3.3.3 we introduced the general structure for a significance test with the comparison of two means in a parallel group trial. This resulted in a procedure which goes under the general heading of the two-sample (or unpaired) t-test. This test was developed for continuous data, although it is applicable more widely and, in particular, is frequently used for score and count data. 

Statistical Thinking for Non-Statisticians in Drug Regulation Richard Kay 2007 John Wiley Sons, Ltd ISBN 978-0-470-31971-0 

There is a connection with what we are seeing here and the calculation of the confidence interval in Chapter 3. Recall Table 3.1 within Section 3.1.3, Changing the multiplying constant . It turns out that p-values and confidence intervals are linked and we will explore this further in a later chapter. The confidence coefficients for d.f. = 38 are 2.02 for 95 per cent confidence and 2.71 for 99 per cent confidence. If we were to look at the tjg distribution we would see that 2.02 cuts off the outer 5 per cent probability while 2.71 cuts off the outer 1 per cent of probability. 

Having calculated the p-value we would also calculate the 95 per cent confidence interval for the difference — P2 to give us information about the magnitude of the treatment effect. For the data in the example in Section 3.3.3 this confidence interval is given by  

So with 95 per cent confidence we can say that the true treatment effect (p — P2) is somewhere in the region 0.2mmHg to 10.6 mmHg. 

---

More generally the test statistic is constructed as the signal/noise (signal-to-noise) ratio or something akin to this. We will develop this methodology in relation the comparison of two independent means for a between-patient design. The resulting test is known as the unpaired t-test or the two-sample t-test. 

As with the unpaired t-test it would be useful to calculate a confidence interval for the treatment effect, p. This is given by ... 

Both of these questions are answered by the unpaired t-test. 

A formal comparison of the two treatments could be based on the unpaired t-test, comparing the mean time to disease recurrence in the test treatment group with the mean time to disease recurrence in the control group. While this is a valid test, it may not be particularly sensitive. The separation between the two groups is clear, but if we now simply read off the times to disease recurrence on the y-axis we will see considerable overlap between the groups we will have lost some sensitivity by ignoring the size of the primary tumour variable. 

To explain in a little more detail, consider a parallel group trial in which we are comparing two treatment means using the unpaired t-test. The null hypothesis Hg p.1 = p.2 that the treatment means are equal is either true or not true God knows, we don t We mere mortals have to make do with data and on the basis of data we will see either a significant p-value (p < 0.05) or a non-significant p-value (p = NS ). The various possibilities are contained in Table 8.1. 

For the unpaired t-test we need to specify the standard deviation, a, for the primary endpoint, and the level of effect, d, we are looking to detect with say 90 per cent power. 

It is generally true that sample size calculations are undertaken based on simple test procedures, such as the unpaired t-test or the test. In dealing with both continuous and binary data it is likely that the primary analysis will ultimately be based on adjusting for important baseline prognostic factors. Usually such analyses will give higher power than the simple alternatives. These more... 

This link applies also to the p-value from the unpaired t-test and the confidence interval for p, the mean difference between the treatments, and in addition extends to adjusted analyses including ANOVA and ANCOVA and similarly for regression. For example, if the test for the slope b of the regression line gives a significant p-value (at the 5 per cent level) then the 95 per cent confidence interval for the slope will not contain zero and vice versa. 

The treatment means will be compared using the unpaired t-test. If, however, the group standard deviations are significantly different according to the F-test, then the comparison of the means will be based on Welch s form of the unpaired t-test. ... 

We will focus in our development in this section on the unpaired t-test. The constant variance assumption can be assessed by undertaking a test (the so-called F-test) relating to the hypotheses ... 

This visual approach based on inspecting the normal probability plot may seem fairly crude. However, most of the test procedures, such as the unpaired t-test, are what we call robust against departures from normality. In other words, the... 

This test is the non-parametric equivalent of the paired t-test. Recall from Section 11.3 that the paired t-test assumes that the population of differences for each patient follows the normal shape. If this assumption is violated then the paired t-test does not apply although, as with the unpaired t-test, the paired t-test is fairly robust against modest departures from normality. 

It is this specific feature that has led to the development of special methods to deal with data of this kind. If censoring were not present then we would probably just takes logs of the patient survival times and undertake the unpaired t-test or its extension ANCOVA to compare our treatments. Note that the survival times, by definition, are always positive and frequently the distribution is positively skewed so taking logs would often be successful in recovering normality. 

In the next section we will discuss Kaplan-Meier curves, which are used both to display the data and also to enable the calculation of summary statistics. We will then cover the logrank and Gehan-Wilcoxon tests which are simple two group comparisons for censored survival data (akin to the unpaired t-test), and then extend these ideas to incorporate centre effects and also allow the inclusion of baseline covariates. 

We mentioned earlier, in Section 13.1, that if we did not have censoring then an analysis would probably proceed by taking the log of survival time and undertaking the unpaired t-test. The above model simply develops that idea by now incorporating covariates etc. through a standard analysis of covariance. If we assume that InT is also normally distributed then the coefficient c represents the (adjusted) difference in the mean (or median) survival times on the log scale. Note that for the normal distribution, the mean and the median are the same it is more convenient to think in terms of medians. To return to the original scale for survival time we then anti-log c, e, and this quantity is the ratio (active divided by control) of the median survival times. Confidence intervals can be obtained in a straightforward way for this ratio. 

A statistical test that is often appropriate for comparing two groups in terms of a quantitative outcome measure is the unpaired t-test. Other assumptions underpin the use of a t-test (see later) and it is therefore sometimes desirable to use one of the tests primarily intended for use on ordinal data even if the data are quantitative. 

A choice between a t test and the comparable test for ordinal data often presents diflGculty. As well as requiring that the outcome measure be quantitative, a t test requires that the samples should be from a parent population in which the measurement is normally distributed. Testing for normality is difficult if the study is very small. If two groups are being compared, the unpaired t test assumes that the two samples come from... 

The unpaired t-test is an example of a parametric method, which means that it is based on the assumption that the two samples are taken from normal, or approximately normal distributions. Generally, parametric tests should be used where possible because they are more powerful (effectively, more sensitive) than the alternative non-parametric methods [32]. However, significance levels obtained from parametric tests may be inaccurate, and the true power of the test may decrease, if the assumption of normality is poor. The non-parametric alternative to the unpaired t-test is the Mann-Whitney test [32]. In this test, a rank is assigned to each observation (1 = smallest, 2 = next smallest, etc.), and the test statistic is computed from these ranks. Obviously, the test is less sensitive to departures from normality, such as the presence of outliers, since, for example, the rank assigned to the smallest observation will always be 1, no matter how small that observation is. 

The limits of aflatoxin Bl detection by the recA-lux and inoA-lux fusion strains were determined by using various concentrations of untreated and activated compound. The bioluminescence of the culture with chemical added was tested to determine if it was greater than that of the control without chemical. The unpaired t test was used and P<0.005 values in a one-tailed test were considered significant. With this criterion. Table II shows the lowest concentrations of aflatoxin Bl detected. 

---