The Bayesian Approach to Statistics

Nevertheless, what currently passes for frequentist parametric statistics includes a collection of techniques, concepts, and methods from each of these two schools, despite the disagreements between the founders. Perhaps this is because, for the very important cases of the normal distribution and the binomial distribution, the MLE and the UMVUE coincided. Efron (1986) suggested that the emotionally loaded terms (unbiased, most powerful, admissible, etc.) contributed by Neyman, Pearson, and Wald reinforced the view that inference should be based on likelihood and this reinforced the frequentist dominance. Frequentist methods work well in the situations for which they were developed, namely for exponential families where there are minimal sufficient statistics. Nevertheless, they have fundamental drawbacks including  

Inference based on the likelihood function using Fisher s ideas is essentially constructive. That means algorithms can be found to construct the solutions. Efron (1986) refers to the MLE as the original jackknife because it is a tool that can easily be adapted to many situations. The maximum likelihood estimator is invariant under a one-to-one reparameterization. Maximum likelihood estimators are compatible with the likelihood principle. Frequentist inference based on the likelihood function has some similarities with Bayesian inference as well as some differences. These similarities and differences will be explored in Section 3.3. 

Bayesian statistics is based on the theorem first discovered by Reverend Thomas Bayes and published after his death in the paper An Essay Towards Solving a Problem in the Doctrine of Chances by his friend Richard Price in Philosophical Transactions of the Royal Society. Bayes theorem is a very clever restatement of the conditional probability formula. It gives a method for updating the probabilities of unobserved events, given that another related event has occurred. This means that we have a prior probability for the unobserved event, and we update this to get its posterior probability, given the occurrence of the related event. In Bayesian statistics, Bayes theorem is used as the basis for inference about the unknown parameters of a statistical distribution. Key ideas forming the basis of this approach include  

Bayes theorem is the only consistent way to modify our belief about the parameters given the data that actually occurred. A Bayesian inference depends only on the data that occurred, not on the data that could have occurred but did not. Thus, Bayesian inference is consistent with the likelihood principle, which states that if two outcomes have proportional likelihoods, then the inferences based on the two outcomes should be identical. For a discussion of the likelihood principle see Bernardo and Smith (1994) or Pawitan (2001). In the next section we compare Bayesian inference with likelihood inference, a frequentist method of inference that is based solely on the likelihood function. As its name implies, it also satisfies the likelihood principle. 

A huge advantage of Bayesian statistics is that the posterior is always found by a single method Bayes theorem. Bayes theorem combines the information about the parameters from our prior density with the information about the parameters from the observed data contained in the likelihood function into the posterior density. It summarizes our knowledge about the parameter given the data we observed. 

---

O Hagan A. 2001. Uncertainty in toxicological predictions the Bayesian approach to statistics. In Rainbow PS, Hopkin SR Crane M, editors. Forecasting the environmental fate and effects of chemicals. Chichester (UK) John Wiley, p 25—41. 

An alternative to the frequentist approach to statistics is based upon the use of probability to quantify the state of knowledge (or ignorance) regarding a quantity. This view is known as the personalist, subjectivist or Bayesian view (Morgan Henrion, 1990). For consistency throughout the text, we will use the term Bayesian . Unlike a frequentist approach, a Bayesian approach does not require assumptions about repeated trials in order to make... 

With greater sophistication of methods of data acquisition, intuition can play a less important role, and a bayesian philosophical approach becomes more important. In contrast to the classical approach to statistics, which is concerned with the distribution of possible measured values about a unique true value, the bayesian approach is concerned with the distribution of possible true values about the measured value at hand—a concept often greeted with hostility by traditional statisticians. 

If the statistics are poor, the measured distribution of the rare counts over sections of the column can be directly simulated a great many times to obtain the likelihood function suitable for the Bayesian approach to the problem. 

This last point is important. In practice we do not consider that only results studied for a given subgroup are relevant to that subgroup we accept that results carry over to some degree. This is, in fact, a necessary condition for any science of statistics. At the very least we have to be prepared to act as if results from one individual had some relevance to another. In fact, both frequentist and Bayesian approaches to statistics allow ways in which this can be done. The Bayesian, for example, can place an informative prior on possible differences in treatment effects between subgroups. Such a prior would state that she believes with high probability that any possible difference between sexes as regards treatment effects will be fairly small. If the treatment effect... 

The concept of entropy is present in many disciplines. In statistical mechanics, Boltzmann introduced entropy as a measure of the number of microscopic ways that a given macroscopic state can be realized. A principle of nature is that it prefers systems that have maximum entropy. Shannon has also introduced entropy into communications theory, where entropy serves as a measure of information. The role of entropy in these fields is not disputed in the scientific community. The validity, however, of the Bayesian approach to probability theory and the principle of maximum entropy in this, remains controversial. 

The development of tools to ensure the use of such objective behef systems, and computational methods such as Monte Carlo simulation have brought the Bayesian approach to the fore in recent years in areas such as parameter estimation, statistical learning, and statistical decision theory. Here, we focus our attention primarily upon parameter estimation, first restricting our discussion to single-response data. 

The classical or frequentist approach to probability is the one most taught in university conrses. That may change, however, becanse the Bayesian approach is the more easily nnderstood statistical philosophy, both conceptually as well as numerically. Many scientists have difficnlty in articnlating correctly the meaning of a confidence interval within the classical frequentist framework. The common misinterpretation the probability that a parameter lies between certain limits is exactly the correct one from the Bayesian standpoint. 

Two very different approaches to inferential statistics exist the classical or fre-quentist approach and the Bayesian approach. Each approach is used to draw conclusions (or inferences) regarding the magnitude of some unknown quantity, such as the intercept and slope of a dose-response model. The key difference between classical... 

Throughout this book, the approach taken to hypothesis testing and statistical analysis has been a frequentist approach. The name frequentist reflects its derivation from the definition of probability in terms of frequencies of outcomes. While this approach is likely the majority approach at this time, it should be noted here that it is not the only approach. One alternative method of statistical inference is the Bayesian approach, named for Thomas Bayes work in the area of probability. 

The whole procedure, with or without salts, may not be based upon sound statistical principles. Rather than using various object functions, it appears better to use a reliable statistical technique such as the method of maximum likelihood (24) or the Bayesian approach (25), both of which take into account the errors in all experimental observations in a logically justifiable fashion. The various discrepancies and anomalies noted in the present work would be moderated by using either... 

Benichou C, Danan G. Causality assessment in the European pharmaceutical industry presentation of preliminary results of a new method. Drug Inf J 1992 26 589-92. Hutchinson T A. Computerised bayesian ADR assessment. Drug Inf J 1991 25 235 1. Lane DA, Hutchinson TA, Jones JK, Kramer MS, Naranjo CA. A Bayesian Approach to Causality Assessment. Universcity of Minnesota School of Statistics Tech Reps No 472 (no date available). 

In Chapter 4 we shall look at inferential statistics and try to explain the difference between two major approaches to statistics the frequentist and the Bayesian. Before... 

Spiegelhalter DJ, Freedman LS, Parmar MKB (1994) Bayesian approaches to randomized trials. Journal of the Royal Statistical Society Series A - Statistics in Society 157 357-387. 

Caves, D. W., Herriges, J. A., Train, K. E. and Windle, R. J. A Bayesian approach to combining conditional demand and engineering models of electricity usage. The Review of Economics and Statistics 69(3) (1987),... 

When conducting a quantitative risk assessment the analyst(s) must define its probabilistic basis, and in most cases this means either to use the classical frequency approach or the Bayesian approach. The classical statistical approach is and has been the most commonly used probabilistic basis in health care (Schneider 2006). This approach interprets a probability as the relative fraction of times the event considered occurs if the situation analyzed were hypothetically repeated an infinite munber of times. The imder-lying probability is unknown, and is estimated in the risk analysis. In the alternative, the Bayesian perspective, a probability is a measure of imcertainty about future events and outcomes (consequences), as seen through the eyes ofthe assessor(s) and based on his/her backgroimd information and knowledge at the time of the analysis. Probability is a subjective measure of imcertainty. 

Frankly speaking, Bayes rule involves the manipulation of conditional probabilities. However, there exists a sharp difference in classical and Bayesian approaches. The major difference between the Bayesian and classical approaches to statistical inference is in the definition of probability. The classical approach asserts that the probability of a fair coin tossed, landing heads is 14 after repetitive tests. In contrast to this, as per Bayesian approach, will say that the probability a coin lands heads is Vz expressing a degree of belief, and would argue that based on the symmetry of the coin, there is no reason to think that one side is more likely to come up than the other side. This definition of probability is usually termed subjective probability [1]. The classical approach uses probability to express the frequency of certain types of data to occur over repeated trials. The Bayesian approach uses probability to express belief in a statement about unknown quantities. This will be clearer from the difference between the two approaches which have divided statistical approaches in two clear divisions, i.e. now in statistics there are two schools of thoughts ... 

---