Experiment-wise error rate

The issue of multiplicity is that when performing multiple statistical tests, the error probability associated with the inferences made is inflated. To see this, let us consider a simple situation where one is interested in performing two statistical tests on independent sets of data, each at a significance level of 0.05. Thus, the probability that each of the two tests will be declared significant erroneously (type I error) is 0.05. However, the probability that at least one of the two tests will be declared significant erroneously is 0.0975. The probability that at least one of the tests of interest will be declared significant erroneously is called the experiment-wise error rate. If we perform three 0.05 level tests, the experiment-wise error rate increases to 0.143. In practical terms, this means that if we perform multiple tests and make multiple inferences, each one at a reasonably low error... 

The need to control the experiment-wise error rate may not apply to exploratory analyses. Statisticians often perform formal statistical tests for exploratory purposes. So, no formal hypotheses are stated and no inferences are made based on them. Even though the act of performing formally an exploratory test involves the same steps as inferential testing, it is conceptually different because of the absence of a null hypothesis. The p-value obtained in such a test should be viewed as a measure of the level of inconsistency of the data with the underlying assumptions of the test rather than error probabilities involved in making causal inferences. 

There are several sources of irreproducibility in kinetics experimentation, but two of the most common are individual error and unsuspected contamination of the materials or reaction vessel used in the experiments. An individual may use the wrong reagent, record an instrument reading improperly, make a manipulative error in the use of the apparatus, or plot a point incorrectly on a graph. Any of these mistakes can lead to an erroneous rate constant. The probability of an individual s repeating the same error in two successive independent experiments is small. Consequently, every effort should be made to make sure that the runs are truly independent, by starting with fresh samples, weighing these out individually, etc. Since trace impurity effects also have a tendency to be time-variable, it is wise to check for reproducibility, not only between runs over short time spans, but also between runs performed weeks or months apart. 

The correct application of filters is central to the success of a TS-PFR experiment since it is the slopes obtained by this filtering that are the rates of the reaction. It would be less than wise to apply outdated or inappropriate methods of error handling to data obtained by a new technique that offers so much promise and such different data. 

---