Relationship Between Confidence Intervals and Probability Levels

4 Relationship Between Confidence Intervals and Probability Levels 

Confidence intervals are intimately related to probability levels since levels of statistical significance can be deduced from the values of the limits of two-sided confidence intervals. In the scenario we have discussed so far in this chapter, if the two-sided 95% Cl excludes zero, the difference will attain significance at the p 0.05 level. Imagine this hypothetical result  

Both the lower and the upper limits lie above zero, and hence the confidence interval excludes zero. This statement therefore also tells us that the mean reduction 

This relationship between confidence intervals and probability levels arises from the fact that, in the scenario of interest here, i.e., the possible difference in mean responses to two treatments, the null value is zero. That is, the treatment effect point estimate that would result if the two group means did not differ at all is zero. Thus, if the 95 % Cl excludes zero, there is compelling evidence at the 5 % level that the effect size is not zero, which means that there is compelling evidence that the treatment group means differ. Similarly, if the 99% Cl excludes zero, there is compelling evidence at the 1 % level that the effect size is not zero, which means that there is even stronger evidence that the treatment group means differ. To state the same in reverse, if the 95 % Cl includes zero, then we cannot exclude the possibifity that there is no difference in responses in the two treatments with 95 % certainty. 

---