Frequentist

In frequentist statistics, probability is instead a long-run relative occurrence of some event out of an infinite number of repeated trials, where the event is a possible outcome of the trial. A hypothesis or parameter that expresses a state of nature cannot have a probability in frequentist statistics, because after an infinite number of experiments there can be no uncertainty in the parameter left. A hypothesis or parameter value is either... 

The usual frequentist procedure comprises a number of steps [11] ... 

If we do this over and over again, we will have done the right thing 95% of the time. Of course, we do not yet know the probability that, say, 6 > 5. For this purpose, confidence intervals for 6 can be calculated that will contain the true value of 6 95% of the time, given many repetitions of the experiment. But frequentist confidence intervals are acmally defined as the range of values for the data average that would arise 95% of the time from a single value of the parameter. That is, for normally distributed data. 

The frequentist interval is often interpreted as if it were the Bayesian interval, but it is fundamentally defined by the probability of the data values given the parameter and not the probability of the parameter given the data. 

By contrast, the frequentist view is often contrary to common sense and common scientific practice. The classic example of this is the stopping rule problem [42]. If I am... 

Another aspect in which Bayesian methods perform better than frequentist methods is in the treatment of nuisance parameters. Quite often there will be more than one parameter in the model but only one of the parameters is of interest. The other parameter is a nuisance parameter. If the parameter of interest is 6 and the nuisance parameter is ( ), then Bayesian inference on 6 alone can be achieved by integrating the posterior distribution over ( ). The marginal probability of 6 is therefore... 

In frequentist statistics, by contrast, nuisance parameters are usually treated with point estimates, and inference on the parameter of interest is based on calculations with the nuisance parameter as a constant. This can result in large errors, because there may be considerable uncertainty in the value of the nuisance parameter. 

Frequentist Criteria for Evaluating Estimators, the Sampling Distribution... 

The classical, frequentist approach in statistics requires the concept of the sampling distribution of an estimator. In classical statistics, a data set is commonly treated as a random sample from a population. Of course, in some situations the data actually have been collected according to a probability-sampling scheme. Whether that is the case or not, processes generating the data will be snbject to stochastic-ity and variation, which is a sonrce of uncertainty in nse of the data. Therefore, sampling concepts may be invoked in order to provide a model that accounts for the random processes, and that will lead to confidence intervals or standard errors. The population may or may not be conceived as a finite set of individnals. In some situations, such as when forecasting a fnture value, a continuous probability distribution plays the role of the popnlation. 

Elicitation of jndgment may be involved in the selection of a prior distribution for Bayesian analysis. However, particularly because of developments in Bayesian computing, Bayesian modeling may be useful in data-rich situations. In those situations the priors may contain little prior information and may be chosen in such a way that the results will be dominated by the data rather than by the prior. The results may be acceptable from a frequentist viewpoint, if not actually identical to some frequentist results. 

Bayesian approaches are discussed throughout this book. Unfortunately, because frequentist methods are typically presented in introductory statistics courses, most environmental scientists do not clearly understand the basic premises of Bayesian methods. This lack of understanding could hamper appreciation for Bayesian approaches and delay the adaptation of these valuable methods for analyzing uncertainty in risk assessments. 

The standard tools of statistical inference, including the concept and approaches of constructing a null hypotheses and associated p values, are based on the frequentist view of probability. From a frequentist perspective, the probability of an event is defined as the fraction of times that the event occurs in a very large number of trials (known as a probability limit). Given a hypothesis and data addressing it, the classical procedure is to calculate from the data an appropriate statistic, which is typically... 

This distribution, together with the numerical value of the statistic, allows an assessment of how unusual the data are, assuming that the hypothesis is valid. The p value is the probability that the observed value of the statistic (or values even more extreme) occur. The data are declared significant at a particular level (a) if p < a, the data are considered sufficiently unusual relative to the hypothesis and the hypothesis is rejected. Standard, albeit arbitrary, values of a are taken as 0.05 and 0.01. Let us suppose that a particular data set gives p = 0.02. From the frequentist vantage, this means that, if the hypothesis were true and the whole experiment were to be repeated many times under identical conditions, in only 2% of such trials would the value of the statistic be more unusual or extreme than the value actually observed. One then prefers to believe that the data are not, in fact, unusual and concludes that the assumed hypothesis is untenable. 

Critics of the frequentist approach consider this disturbing. The actual observations in 12 tosses of a coin, 9 heads and 3 tails were observed should not lead to 2 different conclusions dependent only upon the choice of when to stop the experiment (at 12 tosses or at 3 tails). 

The above approach, which was attacked as being too vague to be the starting point of any theory of probability, led eventually to the frequentist approach, where probability was defined in a manner that assigns a numerical value, albeit a value that cannot ever be measured, since it requires an inhnite number of trials... 

Probability can be defined as a limiting case of a frequency ratio, and from this view the various rules of probability can be derived. An alternative approach is an axiomatic one that states that there is a quantity called probability associated with events and that it possesses assigned properties. The former is largely the frequentist point of view, the axiomatic approach is shared by Bayesians and non-Bayesians alike. 

Probability values lie continuously in the range 0 to 1 inclusive, where the endpoints zero and unity are identified with impossibility and certainty, respectively. This follows immediately for the frequentist for the axiomatic approach it is adopted as an axiom, but one imbued with Laplace s commonsense. Any other range could be chosen at the cost of greater difficulty of interpretation. 

For the Bayesian, the relationship is taken as an axiom, but its motivation reflects the real world with the foreshadowing of rules implied by the above frequentist treatment. Given the 2 events or propositions, A and B, then... 

Equation (5.10) is a statement of Bayes theorem. Since the theorem is proved using results or axioms valid for both frequentist and Bayesian views, its use is not limited to Bayesian applications. Note that it relates 2 conditional probabilities where the events A and B are interchanged. 

Confidence intervals are interpreted differently by frequentists and Bayesians. The 95% confidence interval derived by a frequentist suggests that the true value of some parameter (0) will be contained within the interval 95% of the time in an infinite number of trials. Note that each trial results in a different interval because the data are different. This statement is dependent on the assumed conditions under which the calculations were done, e.g., an infinite number of trials and identical conditions for each trial (O Hagan 2001). Nothing can be said about whether or not the interval contains the true 0. 

The classical or frequentist approach to probability is the one most taught in university conrses. That may change, however, becanse the Bayesian approach is the more easily nnderstood statistical philosophy, both conceptually as well as numerically. Many scientists have difficnlty in articnlating correctly the meaning of a confidence interval within the classical frequentist framework. The common misinterpretation the probability that a parameter lies between certain limits is exactly the correct one from the Bayesian standpoint. 

The Bayesian equivalent to the frequentist 90% confidence interval is delineated by the 5th and 95th percentiles of the posterior distribntion. Bayesian confidence intervals for SSD (Figures 5.4 to 5.5), 5th percentile, i.e., HC5 and fraction affected (Figures 5.4 to 5.6) were calculated from the posterior distribution. Thns, the nncer-tainties of both HC and FA are established in 1 consistent mathematical framework FA estimates at the logio HC lead to the intended protection percentage, i.e., M °(logio HCf) = p where p is a protection level. Further full distribution of HC and FA uncertainty can be very easily extracted from posterior distribntion for any level of protection and visualized (Figures 5.5 to 5.7). 

For the normal distribution there are analytical solutions allowing the assessment of both FA and HC using frequentist statistics. In contrast, Bayesian solutions are numerical. This highlights the flexibility of the Bayesian approach since it can easily deal with any distribution, which is not always possible with the frequentist approach. 

Aldenberg and Jaworska (2000) demonstrate that frequentist statistics and the Bayesian approach with noninformative prior results in identical confidence intervals for the normal distribution. Generally speaking, this is more the exception than the rule. 

For those who feel more confident with the frequentist approach and find the Bayesian approach controversial to some extent, it is advantageous that both approaches yield the same answers in this simplest case. This might add confidence in the Bayesian approach for some practitioners. 

Probability The Bayesian or subjective view is that the probability of an event is the degree of belief that a person has, given some state of knowledge, that the event will occur. In the classical or frequentist view, the probability of an event is the frequency of an event occurring given a long sequence of identical and independent trials. 

When multiplicity is present, the usual frequentist approach to the analysis of clinical trial data may necessitate an adjustment to the type I error. Multiplicity may arise for, example. 

Throughout this book, the approach taken to hypothesis testing and statistical analysis has been a frequentist approach. The name frequentist reflects its derivation from the definition of probability in terms of frequencies of outcomes. While this approach is likely the majority approach at this time, it should be noted here that it is not the only approach. One alternative method of statistical inference is the Bayesian approach, named for Thomas Bayes work in the area of probability. 

When a clinical trial has been conducted, the frequentist approach we have discussed in the book leads to certain statistical analyses being conducted. A p-value is calculated which provides information leading to the rejection of the null hypothesis or the failure to reject the null hypothesis. Additionally, the analyses lead to an estimate of the treatment effect and its associated... 

Statistical methods that are based upon analysis of empirical data without prior assumptions about the type and parameter of distributions are typically termed frequentist methods, although sometimes the term classical is used (e.g. Morgan Henrion, 1990 Warren-Hicks Butcher, 1996 Cullen Frey, 1999). However, the term classical is sometimes connotated with thought experiments (e.g. what happens with a roll of a die) as opposed to inference from empirical data (DeGroot, 1986). Therefore, we use the term frequentist . 

---