Null distribution

Therefore, exact tests are considered that can be performed using two different approaches conditional and unconditional. In the first case, the total number of tumors r is regarded as fixed. As a result the null distribution of the test statistic is independent of the common probability p. The exact conditional null distribution is a multivariate hypergeometric distribution. 

The unconditional model treats the sum of all tumors as a random variable. Then the exact unconditional null distribution is a multivariate binomial distribution. The distribution depends on the unknown probability. 

The current subject matter deals with randomness, independence and trend concerning small sets of electrochemical observations or measurements, whose probability distribution is unknown, and the assumption of an even approximately normal distribution would be statistically unsound. Under such circumstances the theoretical null-distribution related to the hypothesis H of lack of randomness, independence and lack of trend, has to be established from the data themselves on the basis of equal probability of all possible data arrangements. 

Table 1. Establishment of the null distribution of ranks and the D-statistic in testing the hypothesis of upward trend in the rate-loss reduction data in Section II.

It is useful to look at this visually. Figure 3.2 plots each of the outcomes on the x-axis with the corresponding probabilities, calculated when the null hypothesis is true, on the y-axis. Note that the x-axis has been labelled according to heads -tails (H —L), the number of heads minus the number of tails. This identifies each outcome uniquely and allows us to express each data value as a difference. More generally we will label this the test statistic, it is the statistic that the p-value calculation is based. The graph and the associated table of probabilities are labelled the null distribution (of the test statistic). 

Figure 3.2 Null distribution for 20 flips of a fair coin...

This calculation of the p-value comes from the probabilities associated with these signal-to-noise ratios and this forms a common theme across many statistical test procedures. In general, the signal-to-noise ratio is again referred to as the test statistic. The distribution of the test statistic when the null hypothesis is true (equal treatments) is termed the null distribution. 

Determine the null distribution of the chosen test statistic, that is, what are the probabilities associated with all the potential values of the test statistic when... 

Pearson calculated the probabilities associated with values of this test statistic when the treatments are the same, to produce the null distribution. This distribution is called the chi-square distribution on one degree of freedom, denoted x i> and is displayed in Figure 4.2. Note that values close to zero have the highest probability. Values close to zero for the test statistic would only result when the Os and the s agree closely, whereas large values are unlikely when the treatments are the same. 

The null distribution for the t-test depended on the number of subjects in the trial. For the chi-square test comparing two proportions, and providing the sample size is reasonably large, this is not the case the null distribution is always x i- As a consequence we become very familiar with x i- The critical value for 5 per cent significance is 3.841 while 6.635 cuts off the outer 1 per cent probability and 10.83 cuts off the outer 0.1 per cent. 

Usually the type I error is fixed at 0.05 (5 per cent). This is because we use 5 per cent as the significance level the cut-off between significance (p < 0.05) and non-significance (p > 0.05). The null distribution tells us precisely what will happen when the null hypothesis is true we veill get extreme values in the tails of that distribution, even when pj = p2. However, when we do see a value in the extreme outer 5 per cent, we declare significant differences and by definition this will occur 5 per cent of the time when Hg is true. 

Critical values for individual tests and confidence intervals are based on the null distribution of i /aL, that is, on the distribution of this statistic when all effects Pi are zero. Lenth proposed a /-distribution approximation to the null distribution, whereas Ye and Hamada (2000) obtained exact critical values by simulation of Pi /ai under the null distribution. From their tables of exact critical values, the upper 0.05 quantile of the null distribution of Pi /aL is CL = 2.156. On applying Lenth s method for the plasma etching experiment and using a = 0.05 for individual inferences, the minimum significant difference for each estimate is calculated to be cl x l = 60.24. Hence, the effects A, AB, and E are declared to be nonzero, based on individual 95% confidence intervals. 

When screening experiments are used, it is generally anticipated that several effects may be nonzero. Hence, one ought to use statistical procedures that are known to provide strong control of error rates. It is not enough to control error rates only under the complete null distribution. This section discusses exact confidence intervals. Size-a tests are considered in Section 5. 

In another example, the null distribution was compared for 2000 pseudodatasets with the real distribution generated from 2000 runs of 10-fold cross validation for ER232. The distribution of prediction accuracy of the real dataset centered around 82% while the pseudo-datasets were near 50% [60] (Figure 6.12). The distribution turned out to be much narrower for the real dataset than... 

Figure 6.11 Assessment of the chance correlation in DF for four datasets. For each graph the null distribution (dashed) is generated from the results of 10-fold cross validation on 2000 pseudodatasets while the real distribution (solid) is derived from 2000 runs of 10-fold cross vahdation for the original dataset.

Considerable work has been focused on determining the asymptotic null distribution of -2 log-likelihood -ILL) when the alternative hypothesis is the presence of two subpopulations. In the case of two univariate densities mixed in an unknown proportion, the distribution of -ILL has been shown to be the same as the distribution of [max(0, Y)f, where Y is a standard normal random variable (28). Work with stochastic simulations resulted in the proposal that -2LL-c is distributed with d degrees of freedom, where d is equal to two times the difference in the number of parameters between the nonmixture and mixture model (not including parameters used for the probability models) and c=(n-l-p- gl2)ln (31). In the expression for c, n is the number of observations, p is the dimensionality of the observation, and g is the number of subpopulations. So for the case of univariate observations (p = 1), two subpopulations (g = 2), and one parameter distinguishing the mixture submodels (not including the mixing parameter), -2LL-(n - 3)/n with two... 

One method to adjust for boundary problems is to use simulation. First, replace the unknown parameters by their parameter estimates under the null model. Second, simulate data under the null model ignoring the variability in treating the unknown parameters as fixed. Third, the data is then fit to both the null and alternative model and the LRT statistic is calculated. This process is repeated many times and the empirical distribution of the test statistic under the null distribution is determined. This empirical distribution can then be compared to mixtures of chi-squared distributions to see which chi-squared distribution is appropriate. 

---