Computational Bayesian Statistics

In this section we introduce the main ideas of computational Bayesian statistics. We show how basing our inferences on a random sample from the posterior distribution has overcome the main impediments to using Bayesian methods. The first impediment is that the exact posterior cannot be found analytically except for a few special cases. The second is that finding the numerical posterior requires a difficult numerical integration, particularly when there is a large number of parameters. 

Computational Bayesian statistics is based on developing algorithms that we can use to draw samples from the true posterior, even when we only know the unsealed version. There are two types of algorithms we can use to draw a sample from the true posterior, even when we only know it in the unsealed form. The first type are direct methods, where we draw a random sample from an easily sampled density, and reshape this sample by only accepting some of the values into the final sample, in such a way that the accepted values constitute a random sample from the posterior. These methods quickly become inefficient as the number of parameters increase. 

The overall goal of Bayesian inference is knowing the posterior. The fundamental idea behind nearly all statistical methods is that as the sample size increases, the distribution of a random sample from a population approaches the distribution of the population. Thus, the distribution of the random sample from the posterior will approach the true posterior distribution. Other inferences such as point and interval estimates of the parameters can be constructed from the posterior sample. For example, if we had a random sample from the posterior, any parameter could be estimated by the corresponding statistic calculated from that random sample. We could achieve any required level of accuracy for our estimates by making sure our random sample from the posterior is large enough. Existing exploratory data analysis (EDA) techniques can be used on the sample from the posterior to explore the relationships between parameters in the posterior. 

---

Understanding Computational Bayesian Statistics. By William M. Bolstad 1... 

The development and implementation of computational methods for drawing random samples from the incompletely known posterior has revolutionized Bayesian statistics. Computational Bayesian statistics breaks free from the limited class of models where the posterior can be found analytically. Statisticians can use observation models, and choose prior distributions that are more realistic, and calculate estimates of the parameters from the Monte Carlo samples from the posterior. Computational Bayesian methods can easily deal with complicated models that have many parameters. This makes the advantages that the Bayesian approach offers accessible to a much wider class of models. 

This book aims to introduce the ideas of computational Bayesian statistics to advanced undergraduate and first-year graduate students. Students should enter the course with some knowledge in Bayesian statistics at the level of Bolstad (2007). This book builds on that background. It aims to give the reader a big-picture overview of the methods of computational Bayesian statistics, and to demonstrate them for some common statistical applications. 

Computational Bayesian statistics is based on drawing a Monte Carlo random sample from the unsealed posterior. This replaces very difficult numerical calculations with the easier process of drawing random variables. Sometimes, particularly for high dimensional cases, this is the only feasible way to find the posterior. 

We have now looked at each of the parameters of a normal fi, separately, assuming the other parameter is a known constant. In fact, this is all we need to do computational Bayesian statistics using the Gibbs sampler since knowing the conditional distribution of each parameter given the other parameters are known is all that is necessary to implement it. This will be shown in Chapter 10. However, we will look at the case where we have normal fi, observations where both parameters are unknown in the next section. 

Billard and Diday Symbolic Data Analysis Conceptual Statistics and Data Mining Dunne A Statistical Approach to Neural Networks for Pattern Recognition Ntzoufras Bayesian Modeling Using WinBUGS Bolstad Understanding Computational Bayesian Statistics... 

Understanding Computational Bayesian statistics / William M. Bolstad. p. cm. 

---