The chi-squared test

A second assumption is that the uncontrolled variation is random. This would not be the case if, for example, there were some uncontrolled factor, such as temperature change, which produced a trend in the results over a period of time. The effect of such uncontrolled factors can be overcome to a large extent by the techniques of randomization and blocking which are discussed in Chapter 7. 

It will be seen that an important part of ANOVA is the application of the T-test. Use of this test (see Section 3.6) simply to compare the variances of two samples depends on the samples being drawn from a normal population. Fortunately, however, the T-test as applied in ANOVA is not too sensitive to departures from normality of distribution. 

In the significance tests so far described in this chapter the data have taken the form of observations which, apart from any rounding off, have been measured on a continuous scale. In contrast, this section is concerned with frequency, i.e. the number of times a given event occurs. For example. Table 2.2 gives the frequencies of the different values obtained for the nitrate ion concentration when 50 measurements were made on a sample. As discussed in Chapter 2, such measurements are usually assumed to be drawn from a population which is normally distributed. The chi-squared test could be used to test whether the observed frequencies differ significantly from those which would be expected on this null hypothesis. 

To test whether the observed frequencies, Oj, agree with those expected, E according some nuii hypothesis, the statistic is calculated  

Since the caicuiation involved in using this statistic to test for normaiity is relatively complicated, it wiii not be described here. (A reference to a worked example is given at the end of the chapter.) The principle of the chi-squared test is more easily understood by means of the following example. 

---

To test whether the results are also statistically significant, a chi-square test for contingency is applied to the results obtained. However, as the values retrieved from the cases shown in Figure 43 are very small, the total number of different hierarchical control levels in that company, were added to that company s total. This led to the categorized results as shown in Table 18, upon which the chi-square test for contingency is applied. 

The Chi-square test statistic is 8,01, giving a significance level of 0.09, implying that there is no statistically significant relation between the results of the different companies, as shown in figure 43. However, from a practical perspective the differences, if they are real, might be of importance, therefore the results and possible interpretations will be discussed in more detail in sub-Sections 7.2.7 and 7.3. 

As in the steady-state case, the implementation of the chi-square test is quite simple, but has its limitations. One can use a more sophisticated technique such as the Generalized Likelihood Ratio (GLR) test. An alternative formulation of the chi-square test is to consider the components of jk separately (this may be useful for failure isolation information). In this case we compute the innovation of the ith measurement as... 

The behavior of the detection algorithm is illustrated by adding a bias to some of the measurements. Curves A, B, C, and D of Fig. 3 illustrate the absolute values of the innovation sequences, showing the simulated error at different times and for different measurements. These errors can be easily recognized in curve E when the chi-square test is applied to the whole innovation vector (n = 4 and a = 0.01). Finally, curves F,G,H, and I display the ratio between the critical value of the test statistic, r, and the chi-value that arises from the source when the variance of the ith innovation (suspected to be at fault) has been substantially increased. This ratio, which is approximately equal to 1 under no-fault conditions, rises sharply when the discarded innovation is the one at fault. 

This behavior allows easy detection of the source of the anomaly. In practice, under normal operation, the only curve processed in real time is E. But whenever the chi-square test detects a global faulty operation, the sequential process leading to curves F, G, H, and I starts, and the innovation at fault is identified. 

Fisher s exact test must be used in preference to the chi-square test when there are small cell sizes. 

Though Fisher s Exact Test is preferable for analysis of most 2x2 contingency tables in toxicology, the chi square test is still widely used and is preferable in a few unusual situations (particularly if cell sizes are large yet only limited computational support is available). 

A 90% confidence level was assigned for the purposes of the chi-square test. 

There are several mathematically different ways to condnct the minimization of S [see Refs. 70-75]. Many programs yield errors of internal consistency (i.e., the standard deviations in the calculated parameters are due to the deviations of the measured points from the calculated function), and do not consider external errors (i.e., the uncertainty of the measured points). The latter can be accommodated by weighting the points by this uncertainty. The overall rehabU-ity of the operation can be checked by the (chi square) test [71], i.e., S (L + N - ) should be in the range 0.5-1.5 for a reasonable consistency between the measured points and the calculated parameters. 

In comparisons of eqnilibrium constants collected from the literature (e.g.. Fig. 4.22 or [47]), or correlations of data for a large number of systems (e.g.. Figs. 4.20-4.23), it is desirable to present both the statistical uncertainty of each point which is often given by the standard deviation (one or several a s) of the point, and the general reliability (statistical significance) of the whole correlation [76], for which the chi-square test offers a deeper insight into the reliability of the experimental results [77]. More advanced statistical tests for systems of our kind have been described by Ekberg [78]. 

The chi-square test for comparing two proportions or rates was developed by Karl Pearson around 1900 and pre-dates the development of the t-tests. The steps involved in the Pearson chi-square test can be set down as follows ... 

The null distribution for the t-test depended on the number of subjects in the trial. For the chi-square test comparing two proportions, and providing the sample size is reasonably large, this is not the case the null distribution is always x i- As a consequence we become very familiar with x i- The critical value for 5 per cent significance is 3.841 while 6.635 cuts off the outer 1 per cent probability and 10.83 cuts off the outer 0.1 per cent. 

The formulation of the chi-square test procedure in the previous section, using observed and expected frequencies, is the standard way in which this particular test is developed and presented in most textbooks. It can be shown, however, that this procedure is akin to a development following the earlier signal-to-noise ratio approach. 

In general it can be shown that the approach following the signal-to-noise ratio method is mathematically very similar to the standard formulation of the chi-square test using observed and expected frequencies, and in practice they will invariably give very similar results. Altman (1991) (Section 10.7.4) provides more detail on this connection. 

The rule of thumb for the use of Fisher s exact test is based on the expected frequencies in the 2x2 contingency table each of these need to be at least five for the chi-square test to be used. In the example, the expected frequencies in each of the two cells corresponding to success are 3.5, signalling that Fisher s exact test should be used. 

In fact, Fisher s exact test could be used under all circumstances for the calculation of the p-value, even when the sample sizes are not small. Historically, however, we tend not to do this Fisher s test requires some fairly hefty combinatorial calculations in large samples to get the null probabilities and in the past this was just too difficult. For larger samples p-values calculated using either the chi-square test or Fisher s exact test will be very similar so we tend to reserve use of Fisher s exact test for only those cases where there is a problem and use the chi-square test outside of that. 

The chi-square test proceeds, as before, by calculating expected frequencies. These are given in Table 4.8... 

In this section we will discuss the extension of the t-tests for continuous data and the chi-square tests for binary, categorical and ordinal data to deal with more than two treatment arms. 

The odds ratio, for the binary outcome (baby admitted to the special baby unit for respiratory distress) is then 11/492 divided by 24/471, giving a value 0.439. The chi-square test comparing the treatments with regard to the rates of admission gave p = 0.02. 

Suppose a polydisperse system is investigated experimentally by measuring the number of particles in a set of different classes of diameter or molecular weight. Suppose further that these data are believed to follow a normal distribution function. To test this hypothesis rigorously, the chi-squared test from statistics should be applied. A simple graphical examination of the hypothesis can be conducted by plotting the cumulative distribution data on probability paper as a rapid, preliminary way to evaluate whether the data conform to the requirements of the normal distribution. 

The primary statistical tests used in the studies described in this text are based on the chi-square tests which are in turn derived from the chi-square distribution which is based on the chi distribution. These tests include the chi-square test for goodness of fit, the chi-square test of independence, and Fisher s Exact Test. There are also corrections to some of the tests that account for small number deviations, Yates Correction for Continuity, and for multiple studies attempting to verify the same procedures or processes, Bonferroni s correction. 

The number of degrees of freedom used with the chi-square distribution associated with the 2-dimensional distribution would be (N- 1)(M- l)-m, where m is the number of independent parameters estimated from the measurements. For the -dimensional case, the degrees of freedom used with the chi-square distribution would be (N1- )(/V, - I)... (Nf - I) - m, where m is the number of independent parameters estimated from the measurements. The steps used in the implementation of the chi-square test of independence are essentially the same as those listed for the chi-square test for goodness of fit. The only difference is that the expected values must be calculated for all NxM cases in the two-dimensional distribution and for all. .. Nk cases in the -dimensional distribution. The expected values for the cells are often arranged in a table that resembles the contingency table or are sometimes included, inside parentheses, within the same cell of the contingency table as the measurement. 

The second type of probability sum calculated from the set of Ps contains only those trial tables that are more extreme in the same direction as the measured contingency table. As with the chi-square test, if this probability is less than or equal to the significance level, a, chosen for the study, then the null hypothesis of no effect is rejected otherwise, the null hypothesis is accepted. This P is referred to as a one-tail or one-sided P-value and its associated test, a one-tailed test. The difficulty in this type of test is to correctly identify the trial tables from the set that are more extreme in the same direction as the measured contingency table. 

The chi-square test is one of the oldest and most venerable statistical tests and you will certainly come across cases where other people have used it. You may also want to use it yourself as you know your readers are likely to be familiar with it. 

The chi-square test is flexible (it can be applied in situations where cases are categorized into more than two classes). 

Throughout this book, it has been emphasized that we should always try to go beyond statistical significance and also consider the extent of any difference and so assess practical significance. Unfortunately the chi-square test does not produce a 95 per cent Cl for the extent of the difference. However, so long as we are only considering a 2x2 table, some packages (including Minitab) provide a separate routine to calculate a confidence interval. 

The chi-square test only assesses statistical significance. To assess practical significance, you will need a separate routine to calculate a 95 per cent Cl for the difference between proportions. 

Ideally the total number allocated to each approach would have been the same. However, this was not quite achieved. This is not a problem, because the chi-square test takes account of different column totals . For optimum power, extreme variation in group sizes should be avoided. 

Statistical analysis of the data is performed by means of the Student s t-test for paired or unpaired data, or by means of analysis of variance followed by the Tukey test. Statistical analysis of nonparametric data is made by the chi square test. 

There are two important statistical tests which can be used to determine whether the differences between two sets of data are real and significant or are just due to chance errors. Both assume that the experimental results are independently and normally distributed. One is the t-test and the other is the chi-squared test. The t-test applies only to continuous data-usually a measurement. The chi-squared test in many cases can approximate frequency or count type of data. 

---