Gibbs-sampling

Suppose we decide to use the true conditional density as the candidate density at each step for every block of parameters given the others. In that case 

Example 9 (continued) Suppose we look at the same bivariate target density as before. The target density is given by 

We can use any of these methods to draw a sample from the posterior. However, the samples won t be random. Draws from a Markov chain are serially correlated. 

We would like the samples to be random, at least approximately, in order to do our inferences. In Chapter 7 we will look at how we can obtain an approximately random sample from the nonrandom sample we have obtained from the Markov chain. 

There are many candidate distributions we can choose from. Each of them would have its own mixing properties. We would like to choose a chain that has good mixing properties. In other words, a chain that moves through the parameter space quickly. The trace plots give an indication of the mixing properties. 

---

If draws can be made from the posterior distribution for each component conditional on values for the others, i.e., fromp(Q,i y, 6,- J, then this conditional posterior distribution can be used as the proposal distribution. In this case, the probability in Eq. (23) is always 1, and all draws are accepted. This is referred to as Gibbs sampling and is the most common form of MCMC used in statistical analysis. 

Maximum likelihood methods used in classical statistics are not valid to estimate the 6 s or the q s. Bayesian methods have only become possible with the development of Gibbs sampling methods described above, because to form the likelihood for a full data set entails the product of many sums of the form of Eq. (24) ... 

A straightforward Gibbs sampling strategy when the number of components is known (or fixed) is as follows [48]. 

A partially Bayesian approach was suggested by Chipman et al. (1997). They used independent prior distributions for each main effect being active. The prior distribution selected for Pj was a mixture of normals, namely, N(0, r ) with prior probability 1 — tzj and N(0, Cj if) with prior probability ttj, where Cj greatly exceeds 1. The prior distribution for a2 was a scaled inverse-x2. They then used the Gibbs-sampling-based stochastic search variable selection method of George and McCulloch (1993) to obtain approximate posterior probabilities for Pj, that is, for each factor they obtained the posterior probability that Pj is from /V(0, cj if) rather than from N(0, r ). They treated this as a posterior probability that the corresponding factor is active and used these probabilities to evaluate the posterior probability of each model. 

George, E. I. and McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88, 881-889. 

Dellaportas, P. and Smith, A. F. M. (1993). Bayesian inference for generalized linear and proportional hazards models via Gibbs sampling. Applied Statistics, 42, 443 159. 

BUGS (Bayesian inference nsing Gibbs sampling) mathstat.helsinki.fi/ openbugs/ Bayesian analysis of complex statistical models nsing MCMC... 

Spiegelhalter, D.J., A. Thomas, N.G. Best, and W. R. Gilks. 1996. BUGS Bayesian Inference Using Gibbs Sampling, Version 0.5, (version ii). Online. Available http //www.nirc-bsu.cam.ac.uk/bugs/... 

Bryant Gibbs Threading Gibbs sampling for sequence-structure alignment with contact potentials... 

A specific, in some sense the simplest Markov Chain Monte Carlo algorithm (—> Monte Carlo methods). In Gibbs sampling one samples the variables of a solution space one after the other. Each variable is drawn from a conditional distribution on the values of the other variables that were pre-... 

Neuwald, A. F. Wootton, J. C. (1993). Detecting subtle sequence signals a Gibbs sampling strategy for multiple alignment. Science 262(5131), 208-14. 

Gibbs DNA or protein motifs using Gibbs sampling http //bayesweb.wadsworth.org/gibbs/gibbs.html... 

CompareProspector DNA motifs in eukaryotes using biased Gibbs sampling requires multiple alignment http // seqmotifs. Stanford. edu... 

Siddharthan, R., Siggia, E. D., and van Nimwegen, E. (2005) PhyloGibbs a gibbs sampling motif finder that incorporates phylogeny. PLoS Comput. Biol. 1, e67. 

Thijs, G., Marchal, K., Lescot, M., et al. (2002) A Gibbs sampling method to detect overrepresented motifs in the upstream regions of coexpressed genes. J. Comput. Biol. 9, 447 164. 

Key Words Gibbs sampling phylogenetic footprinting transcription regulation. 

---