Distribution , posterior

Most often the hypothesis H concerns the value of a continuous parameter, which is denoted 0. The data D are also usually observed values of some physical quantity (temperature, mass, dihedral angle, etc.) denoted y, usually a vector, y may be a continuous variable, but quite often it may be a discrete integer variable representing the counts of some event occurring, such as the number of heads in a sequence of coin flips. The expression for the posterior distribution for the parameter 0 given the data y is now given as... 

The dashed line gives the posterior distribution of 0iej, the solid line gives the prior distribution, and the dotted line gives the likelihood of the data. The figure shows that the posterior is a compromise between the prior and the likelihood. We can integrate the posterior distribution to decide the amount of the bet from... 

Figure 1 (-------) Prior, ) data (likelihood), and (—) posterior distributions for estimating a...

Another aspect in which Bayesian methods perform better than frequentist methods is in the treatment of nuisance parameters. Quite often there will be more than one parameter in the model but only one of the parameters is of interest. The other parameter is a nuisance parameter. If the parameter of interest is 6 and the nuisance parameter is ( ), then Bayesian inference on 6 alone can be achieved by integrating the posterior distribution over ( ). The marginal probability of 6 is therefore... 

In the next subsection, I describe how the basic elements of Bayesian analysis are formulated mathematically. I also describe the methods for deriving posterior distributions from the model, either in terms of conjugate prior likelihood forms or in terms of simulation using Markov chain Monte Carlo (MCMC) methods. The utility of Bayesian methods has expanded greatly in recent years because of the development of MCMC methods and fast computers. I also describe the basics of hierarchical and mixture models. 

Calculating the posterior distribution for the parameters given existing data. This calculation can sometimes be perfonned analytically, but in the general case it is performed by simulation. 

Evaluating the model in tenns of how well the model fits the data, including the use of posterior predictive simulations to determine whether data predicted from the posterior distribution resemble the data that generated them and look physically reasonable. Overfitting the data will produce unrealistic posterior predictive distributions. 

We need a mathematical representation of our prior knowledge and a likelihood function to establish a model for any system to be analyzed. The calculation of the posterior distribution can be perfonned analytically in some cases or by simulation, which I... 

We can think of the beta distribution as the likelihood of a prior successes and (3 failures out of a + (3 experiments. The F functions in front serve as a normalization constant, so that li/>(0) dQ =. Note that for an integer, Y x + ) = xl The posterior distribution that results from multiplying together the right-hand sides of Eqs. (2) and (3) is also a beta distribution ... 

We can see that the prior and posterior distributions have the same mathematical forms, as is required of conjugate functions. Also, we have an analytical form for the posterior, which is exact under the assumptions made in the model. 

Again, the use of the conjugate prior distribution results in the analytical form of the posterior distribution and therefore also simple expressions for the expectation values for the 6 , their variances, covariances, and modes ... 

In practice, it may not be possible to use conjugate prior and likelihood functions that result in analytical posterior distributions, or the distributions may be so complicated that the posterior cannot be calculated as a function of the entire parameter space. In either case, statistical inference can proceed only if random values of the parameters can be drawn from the full posterior distribution ... 

If we could draw directly from the posterior distribution, then we could plot p(Q y) from a histogram of the draws on 6. Similarly, we could calculate the expectation value of any function of the parameters by making random draws of 6 from the posterior distribution and calculating... 

In some cases, we may not be able to draw directly from the posterior distribution. The difficulty lies in calculating the denominator of Eq. (18), the marginal data distribution p(y). But usually we can evaluate the ratio of the probabilities of two values for the parameters, p(Q, y)/p(Qu y), because the denominator in Eq. (18) cancels out in the ratio. The Markov chain Monte Carlo method [40] proceeds by generating draws from some distribution of the parameters, referred to as the proposal distribution, such that the new draw depends only on the value of the old draw, i.e., some function We accept... 

If draws can be made from the posterior distribution for each component conditional on values for the others, i.e., fromp(Q,i y, 6,- J, then this conditional posterior distribution can be used as the proposal distribution. In this case, the probability in Eq. (23) is always 1, and all draws are accepted. This is referred to as Gibbs sampling and is the most common form of MCMC used in statistical analysis. 

Step 2. Draw a value for each 6 = jj. , aj from the normal posterior distribution for Aj data points with average yi. 

There is some confusion in using Bayes rule on what are sometimes called explanatory variables. As an example, we can try to use Bayesian statistics to derive the probabilities of each secondary structure type for each amino acid type, that is p( x r), where J. is a, P, or Y (for coil) secondary strucmres and r is one of the 20 amino acids. It is tempting to writep( x r) = p(r x)p( x)lp(r) using Bayes rule. This expression is, of course, correct and can be used on PDB data to relate these probabilities. But this is not Bayesian statistics, which relate parameters that represent underlying properties with (limited) data that are manifestations of those parameters in some way. In this case, the parameters we are after are 0 i(r) = p( x r). The data from the PDB are in the form of counts for y i(r), the number of amino acids of type r in the PDB that have secondary structure J.. There are 60 such numbers (20 amino acid types X 3 secondary structure types). We then have for each amino acid type a Bayesian expression for the posterior distribution for the values of xiiry. 

The Bayesian alternative to fixed parameters is to define a probability distribution for the parameters and simulate the joint posterior distribution of the sequence alignment and the parameters with a suitable prior distribution. How can varying the similarity matrix... 

Zhu et al. [15] and Liu and Lawrence [61] formalized this argument with a Bayesian analysis. They are seeking a joint posterior probability for an alignment A, a choice of distance matrix 0, and a vector of gap parameters. A, given the data, i.e., the sequences to be aligned p(A, 0, A / i, R2). The Bayesian likelihood and prior for this posterior distribution is... 

Unfortunately, some authors describing their work as Bayesian inference or Bayesian statistics have not, in fact, used Bayesian statistics rather, they used Bayes rule to calculate various probabilities of one observed variable conditional upon another. Their work turns out to comprise derivations of informative prior distributions, usually of the form piQi, 02,..., 0 1 = which is interpreted as the posterior distribution... 

A similar formalism is used by Thompson and Goldstein [90] to predict residue accessibilities. What they derive would be a very useful prior distribution based on multiplying out independent probabilities to which data could be added to form a Bayesian posterior distribution. The work of Arnold et al. [87] is also not Bayesian statistics but rather the calculation of conditional distributions based on the simple counting argument that p(G r) = p(a, r)lp(r), where a is some property of interest (secondary structure, accessibility) and r is the amino acid type or some property of the amino acid type (hydro-phobicity) or of an amino acid segment (helical moment, etc). 

For the Qijtnab we use Dirichlet priors combined with a multinominal likelihood to determine a Dirichlet posterior distribution. The data in this case are the set of counts riijuiab -We detennined these counts from PDB data (lists of values for ( ), V /, %i, X2> X3> XA) by counting side chains in overlapping 20° X 20° square blocks centered on (( )a, fb) spaced 10° apart. The likelihood is therefore of the fonn... 

As an example of analysis of side-chain dihedral angles, the Bayesian analysis of methionine side-chain dihedrals is given in Table 3 for the ri = rotamers. In cases where there are a large number of data—for example, the (3, 3, 3) rotamer—the data and posterior distributions are essentially identical. These are normal distributions with the averages and standard variations given in the table. But in cases where there are few data. 

Figure 2 Data distribution and draws from the posterior distribution (mu sim) and posterior predictive distributions (data sim) for methionine side chain dihedral angles. The results for three ro-tamers are shown, (ri = 3 r2 = 1 = 3), (3, 2, 3), and (3, 3, 3). Each simulation consisted of...

There are two special cases for which equations 2.6-7 and 2.6-8 are easily solved to fold a prior distribution with the update distribution to obtain a posterior distribution with the sarai rm as the prior distribution. These distributions are the Bayes conjugates shown in Table 2.6-1. 

To obtain the confidence bounds, the posterior distribution (equation 2.6-12) is integrated from zero to A, , where A is the upper... 

A number of issues arise in using the available data to estimate (he rates of location-dependent fire occurrence. These include the possible reduction in the frequency of fires due to increased awareness. Apostolakis and Kazarians (1980) use the data of Table 5.2-1 and Bayesian analysis to obtain the results in Table 5.2-2 using conjugate priors (Section 2.6.2), Since the data of Table 5.2-1 are binomially distributed, a gamma prior is used, with a and P being the parameters of the gamma prior as presented inspection 2.6.3.2. For example, in the cable- spreading room fromTable 5.2-2, the values of a and p (0.182 and 0.96) yield a mean frequency of 0.21, while the posterior distribution a and p (2.182 and 302,26) yields a mean frequency of 0.0072. 

The 5 percentile Poos tuid Uie 95 percentile P095 of tlie posterior distribution of z are defined by... 

Comparison of the 5" and 95 percentiles of the posterior distribution of Z with the 5 and 95 percentiles of the prior distribution of Z indicates tliat the posterior pdf lies to the left of the prior pdf. Therefore, the posterior pdf assigns higher probability to intervals in the lower part of the range of z than the prior pdf. Tliis reflects the influence of tlie observed occurrence of no failures in 10 years. 

The mean of tlie posterior distribution of Z is Bayesian estimate of the failure rate per year. If E(Z B) is tlie mean of the posterior distribution, then... 

The method for estimating parameters from Monte Carlo simulation, described in mathematical detail by Reilly and Duever (in preparation), uses a Bayesian approach to establish the posterior distribution for the parameters based on a Monte Carlo model. The numerical nature of the solution requires that the posterior distribution be handled in discretised form as an array in computer storage using the method of Reilly 2). The stochastic nature of Monte Carlo methods implies that output responses are predicted by the model with some amount of uncertainty for which the term "shimmer" as suggested by Andres (D.B. Chambers, SENES Consultants Limited, personal communication, 1985) has been adopted. The model for the uth of n experiments can be expressed by... 

---