Comparison of two experimental means

The use in significance testing of critical values from statistical tables was adopted because it was formerly too tedious to calculate the probability of t exceeding the experimental value. Computers have altered this situation, and statistical software usually quotes the results of significance tests in terms of a probability. If the individual data values are entered in Minitab, the result of performing this test is shown below. 

Another way in which the results of a new analytical method may be tested is by comparing them with those obtained by using a second (perhaps a reference) method. In this case we have two sample means and X2. Taking the null hypothesis that the two methods give the same result, that is Hq Pi = P2/ we need to test whether (Xj - X2) differs significantly from zero. If the two samples have standard deviations which are not significantly different (see Section 3.5 for a method of testing this assumption), a pooled estimate, s, of the standard deviation can be calculated from the two individual standard deviations Sj and 2- 

In order to decide whether the difference between two sample means x and X2 is significant, that is to test the null hypothesis, Hq Pi = 1 2, the statistic t is calculated  

This method assumes that the samples are drawn from populations with equal standard deviations. 

In a comparison of two methods for the determination of chromium in rye grass, the following results (mg kg Cr) were obtained  

---

In some circumstances, the use of equation (1) may not be appropriate for the comparison of two experimental means. Examples of when this may be the case are if... 

To illustrate the application of this concept to the two-sample problem, consider the comparison of two population means. For example, suppose a chemical agent was added to the feed of one group of mice while a second group had a chemical free diet. Suppose further that one wished to assess the effect, if any, of the chemical on body weight. Based on the experimental data, one could construct a confidence interval for the difference of the two mean values. By virtue of the discussion above, if this confidence interval contained the value 0, one could conclude that there was no difference in the mean values. 

From a statistical viewpoint it is difficult to state a definite answer because several problems accumulate here. First, there is not an exact solution to the comparison of two population means, for which we have to estimate simultaneously (from the limited experimental data available) their average values and their associated variances, as is the case in laboratories. Second, the equations stated above were deduced for normal distributions but the slopes derived from a least-squares fit follow approximately a Student s distribution. We must bear in mind that although the theoretical slope and intercept of the population follow a normal distribution their estimators do not because the latter (along with the variance of the regression itself) must be estimated from (usually) a very reduced number of data and the number of degrees of freedom - dof- must be taken into account. In statistical terms an intermediate pivot statistic must be introduced to obtain an approximate Student s distribution. ... 

A systematic comparison of two sets of data requires a numerical evaluation of their likeliness. TOF-SARS and SARIS produce one- and two-dhnensional data plots, respectively. Comparison of sunulated and experimental data is accomplished by calculating a one- or two-dimensional reliability (R) factor [33], respectively, based on the R-factors developed for FEED [34]. The R-factor between tire experimental and simulated data is minimized by means of a multiparameter simplex method [33]. 

Fig. 16. Comparison of the experimental IF vibrational distribution from the reaction F + I2 (ref. 554) with the distribution derived from trajectory calculations using a LEPS surface. The trajectory results are represented as histograms with the hatched regions being two standard derivations centred around the mean vibrational population. (Reproduced from ref. 562 by permission of the authors and the Royal Society of Chemistry.)...

Specific examples of hypothesis tests that chemists often use include the comparison of (1) the mean of an experimental data set with what is believed to be the true value (2) the mean to a predicted or cutoff (threshold) value and (3) the means or the standard deviations from two or more sets of data. The sections that follow consider some of the methods for making these comparisons. 

Having established that the standard deviations of two sets of data agree at a reasonable confidence level it is possible to proceed to a comparison of the mean results derived from the two sets, using the t-test in one of its forms. As in the previous case, the factor is calculated from the experimental set of results and compared with the table of critical values (Table 2.3). If /jX ) exceeds the critical value for the appropriate number of degrees of freedom, the difference between the means is said to be significant. When there is an accepted value for the result based on extensive previous analysis t is computed from equation (2.9)... 

Descriptions of the experimental scattering and microscopy conditions have been published elsewhere and are referenced in each section. Throughout this report certain conventions will be used when describing uncertainties in measurements. Plots of small angle scattering data have been calculated from circular averaging of two-dimensional files. The uncertainties are calculated as the estimated standard deviation of the mean. The total combined uncertainty is not specified in each case since comparisons are made with data obtained under... 

---