Fisher randomization test

Rubin DB (1980) Randomization analysis of experimental data - the Fisher randomization test -Comment. Journal of the American Statistical Association 75 591-593. 

To compare the variances of two random samples or their standard deviations, Fishers f-test is applied. The f-value is calcu-... 

Value of coordination. A handful of papers (e.g. Chandra and Fisher 1994, Fumero and Vercellis 1999) have studied the value of coordination between production and distribution based on computational experiments on random test problems. It would be very interesting if one could quantify the value of coordination theoretically (e.g. worst-case or average-case analysis) for certain models. This could then be used to decide whether it is worth the effort to consider production and distribution jointly. 

Unfortunately, it is impossible to design an experiment that will totally disprove a theory based on random phenomena. Various outcomes may occur, some of which may be unlikely but not impossible. Thus Popper s falsifiability condition does not apply. The statistical method advocated by Fisher (1956) attempts to overcome this problem by substituting unlikely for impossible but otherwise follows the principles of the scientific method. With this substitution, Fisher and others proposed conceptual structures for testing theories and scientific hypotheses under conditions of uncertainty that are analogous to the scientific method. However, these approaches, although being very useful in practice, have raised a host of conceptual issues that are the subject of ongoing debates. 

The objective interpretation of probability calculus (Popper, 1976 48, and Appendix IX, Third Comment [1958]) is necessary because no result of statistical sampling is ever inconsistent with a statistical theory unless we make them with the help of. .. rejection rules (Lakatos, 1974 179 see also Nagel, 1971 366). It is under these rejection rules that probability calculus and logical probability approach each other these are also the conditions under which Popper explored the relationship of Fisher s likelihood function to his degree of corroboration, and the conditions arise only if the random sample is large and (e) is a statistical report asserting a good fit (Farris et ah, 2001). In addition to the above, in order to maintain an objective interpretation of probability calculus, Popper also required that once the specified conditions are obtained, we must proceed to submit (e) itself to a critical test, that is, try to find observable states of affairs that falsify (e). 

Having said all of this, it is important to remember, however (Popper, 1976 Appendix IX), ... that non-statistical theories have as a rule a form totally different from that of the h here described, that is, they are of the form of a universal proposition. The question thus becomes whether systematics, or phylogeny reconstruction, can be construed in terms of a statistical theory that satisfies the rejection criteria formulated by Popper (see footnote 1) and that, in case of favorable evidence, allows the comparison of degree of corroboration versus Fisher s likelihood function. As far as phylogenetic analysis is concerned, I found no indication in Popper s writing that history is subject to the same logic as the test of random samples of statistical data. As far as a metric for degree of corroboration relative to a nonstatistical hypothesis is concerned. Popper (1973 58-59 see also footnote 1) clarified. 

The universally accepted approach to statistical evaluation of small data sets is that originated by WUham Gosset (Gosset 1908) and developed and publicized by Richard Fisher (Fisher 1925, 1990). The so-caUed Student s t-distribution (see the accompanying text box) addresses the problem of estimating the uncertainty in the mean of an assumed normal distribution based on a small number of data points assumed to be random samples from that distribution. It forms the basis of f-tests commonly used for the statistical significance of the difference between the means of two small data sets believed to be drawn from the same normal distribution, and for determination of the confidence interval for the difference between two such data set means. 

If the two choices are not independent (i.e., subjects must accept one of two choices and reject the other), then data should be analyzed as frequencies. For simple choice tests, the G-test or Fisher s exact test (Sokal Rohlf 1995) are often used to test for deviations of the observed pattern of choices from a random pattern. The choice of test depends on sample size and calculated expected values. Sometimes, the proportion of subjects on the test substrate is calculated as T T + C), where T is the number of subjects on the test treatment and C is the number of subjects on the control treatment. These proportions often follow a binomial distribution and can be analyzed as continuous variables after employing the arcsine square root transformation. However, analysis of frequencies is usually preferred to analysis of proportions (Sokal Rohlf 1995). 

---