Chi Square test

Crowe, C.M., Recursive Identification of Gross Errors in Linear Data Reconciliation, AJChE Journal, 34(4), 1988,541-550. (Global chi square test, measurement test)... 

If the normal approximation to the binomial distribution is valid (that is, not more than 20% of expected cell counts are less than 5) for drug therapy and symptom of headache, then you can use the Pearson chi-square test to test for a difference in proportions. To get the Pearson chi-square / -value for the preceding 2x2 table, you run SAS code like the following ... 

Here you can still use the Pearson chi-square test as shown in the 2x2 table example as long as your response variable is nominal and merely descriptive. If your response variable is ordinal, meaning that it is an ordered sequence, and you can use a parametric test, then you should use the Mantel-Haenszel test statistic for parametric tests of association. For instance, if in our previous example the variable called headache was coded as a 2 when the patient experienced extreme headache, a 1 if mild headache, and a 0 if no headache, then headache would be an ordinal variable. You can get the Mantel-Haenszel /pvalue by running the following SAS code ... 

To test whether the results are also statistically significant, a chi-square test for contingency is applied to the results obtained. However, as the values retrieved from the cases shown in Figure 43 are very small, the total number of different hierarchical control levels in that company, were added to that company s total. This led to the categorized results as shown in Table 18, upon which the chi-square test for contingency is applied. 

The Chi-square test statistic is 8,01, giving a significance level of 0.09, implying that there is no statistically significant relation between the results of the different companies, as shown in figure 43. However, from a practical perspective the differences, if they are real, might be of importance, therefore the results and possible interpretations will be discussed in more detail in sub-Sections 7.2.7 and 7.3. 

If H0 is rejected, a two-stage procedure is initiated. First, a list of candidate biases and leaks is constructed by means of the recursive search scheme outlined by Romagnoli (1983). All possible combinations of gross errors (measurement biases and/or process leaks) from this subset are analyzed in the second stage. Gross error magnitudes are estimated simultaneously for each combination and chi-square test statistic calculations are performed to identify the suspicious combinations. We will now explain the stages of the procedure. 

As in the steady-state case, the implementation of the chi-square test is quite simple, but has its limitations. One can use a more sophisticated technique such as the Generalized Likelihood Ratio (GLR) test. An alternative formulation of the chi-square test is to consider the components of jk separately (this may be useful for failure isolation information). In this case we compute the innovation of the ith measurement as... 

The behavior of the detection algorithm is illustrated by adding a bias to some of the measurements. Curves A, B, C, and D of Fig. 3 illustrate the absolute values of the innovation sequences, showing the simulated error at different times and for different measurements. These errors can be easily recognized in curve E when the chi-square test is applied to the whole innovation vector (n = 4 and a = 0.01). Finally, curves F,G,H, and I display the ratio between the critical value of the test statistic, r, and the chi-value that arises from the source when the variance of the ith innovation (suspected to be at fault) has been substantially increased. This ratio, which is approximately equal to 1 under no-fault conditions, rises sharply when the discarded innovation is the one at fault. 

This behavior allows easy detection of the source of the anomaly. In practice, under normal operation, the only curve processed in real time is E. But whenever the chi-square test detects a global faulty operation, the sequential process leading to curves F, G, H, and I starts, and the innovation at fault is identified. 

This is called the truncated chi-squares test (Tong and Crowe, 1995). 

Fisher s exact test must be used in preference to the chi-square test when there are small cell sizes. 

Though Fisher s Exact Test is preferable for analysis of most 2x2 contingency tables in toxicology, the chi square test is still widely used and is preferable in a few unusual situations (particularly if cell sizes are large yet only limited computational support is available). 

The RxC chi-square test can be used to analyze discontinuous (frequency) data as in the Fisher s exact or 2x2 chi-square tests. However, in the RxC test (R = row, C = Column) we wish to compare three or more sets of data. An example would be comparison of the incidence of tumors among mice on three or more oral dosage levels. We can consider the data as positive (tumors) or negative (no tumors). The expected frequency for any box is equal to (row total)(column total)/(A/,otal). 

Typically, comparison of incidences of any one type of lesion between controls and treated animals are made using the multiple 2x2 chi square test or Fisher s exact test with a modification of the numbers of animals as the denominators. Too often, experimenters exclude from consideration all those animals (in both groups) that died prior to the first animals being found with a lesion at that site. 

A 90% confidence level was assigned for the purposes of the chi-square test. 

There are several mathematically different ways to condnct the minimization of S [see Refs. 70-75]. Many programs yield errors of internal consistency (i.e., the standard deviations in the calculated parameters are due to the deviations of the measured points from the calculated function), and do not consider external errors (i.e., the uncertainty of the measured points). The latter can be accommodated by weighting the points by this uncertainty. The overall rehabU-ity of the operation can be checked by the (chi square) test [71], i.e., S (L + N - ) should be in the range 0.5-1.5 for a reasonable consistency between the measured points and the calculated parameters. 

In comparisons of eqnilibrium constants collected from the literature (e.g.. Fig. 4.22 or [47]), or correlations of data for a large number of systems (e.g.. Figs. 4.20-4.23), it is desirable to present both the statistical uncertainty of each point which is often given by the standard deviation (one or several a s) of the point, and the general reliability (statistical significance) of the whole correlation [76], for which the chi-square test offers a deeper insight into the reliability of the experimental results [77]. More advanced statistical tests for systems of our kind have been described by Ekberg [78]. 

A test of the null h)rpothesis that the rates of infection are equal - Hq x jii/hnj = 1 gives a p-value of 0.894 using a chi-squared test. There is therefore no statistical evidence of a difference between the treatments and one is unable to reject the null hypothesis. However, the contrary statement is not true that therefore the treatments are the same. As Altman and Bland succinctly put it, absence of evidence is not evidence of absence. The individual estimated infection rates are jTi = 0.250 and = 0.231 that gives an estimated RR of 0.250/0.231 = 1.083 with an associated 95% confidence interval of 0.332-3.532. In other words, inoculation can potentially reduce the infection by a factor of three, or increase it by a factor of three with the implication that we are not justified in claiming that the treatments are equivalent. 

Some statistical tests are specific for evaluation of normality (log-normality, etc., normality of a transformed variable, etc.), while other tests are more broadly applicable. The most popular test of normality appears to be the Shapiro-Wilk test. Specialized tests of normality include outlier tests and tests for nonnormal skewness and nonnormal kurtosis. A chi-square test was formerly the conventional approach, but that approach may now be out of date. 

As we shall see later the data type to a large extent determines the class of statistical tests that we undertake. Commonly for continuous data we use the t-tests and their extensions analysis of variance and analysis of covariance. For binary, categorical and ordinal data we use the class of chi-square tests (Pearson chi-square for categorical data and the Mantel-Haenszel chi-square for ordinal data) and their extension, logistic regression. 

The chi-square test for comparing two proportions or rates was developed by Karl Pearson around 1900 and pre-dates the development of the t-tests. The steps involved in the Pearson chi-square test can be set down as follows ... 

The null distribution for the t-test depended on the number of subjects in the trial. For the chi-square test comparing two proportions, and providing the sample size is reasonably large, this is not the case the null distribution is always x i- As a consequence we become very familiar with x i- The critical value for 5 per cent significance is 3.841 while 6.635 cuts off the outer 1 per cent probability and 10.83 cuts off the outer 0.1 per cent. 

The formulation of the chi-square test procedure in the previous section, using observed and expected frequencies, is the standard way in which this particular test is developed and presented in most textbooks. It can be shown, however, that this procedure is akin to a development following the earlier signal-to-noise ratio approach. 

In general it can be shown that the approach following the signal-to-noise ratio method is mathematically very similar to the standard formulation of the chi-square test using observed and expected frequencies, and in practice they will invariably give very similar results. Altman (1991) (Section 10.7.4) provides more detail on this connection. 

Pearson s chi-square test is what we refer to as a large sample test this means that provided the sample sizes are fairly large then it works well. Unfortunately when the sample sizes in the treatment groups are not large there can be problems. Under these circumstances we have an alternative test, Fisher s exact test. 

We observed the 6/1 split in terms of successes in our data and we can calculate the p-value by adding up the probabilities associated with those outcomes which are as extreme, or more extreme, than what we have observed, when the null hypothesis is true (equal treatments). This gives p = 0.097. The corresponding chi-square test applied (inappropriately) to these data would have givenp = 0.041, so the conclusions are potentially impacted by this. 

The rule of thumb for the use of Fisher s exact test is based on the expected frequencies in the 2x2 contingency table each of these need to be at least five for the chi-square test to be used. In the example, the expected frequencies in each of the two cells corresponding to success are 3.5, signalling that Fisher s exact test should be used. 

In fact, Fisher s exact test could be used under all circumstances for the calculation of the p-value, even when the sample sizes are not small. Historically, however, we tend not to do this Fisher s test requires some fairly hefty combinatorial calculations in large samples to get the null probabilities and in the past this was just too difficult. For larger samples p-values calculated using either the chi-square test or Fisher s exact test will be very similar so we tend to reserve use of Fisher s exact test for only those cases where there is a problem and use the chi-square test outside of that. 

---