Normal distribution conjugate prior

Step 1. From a histogram of the data, partition the data into N components, each roughly corresponding to a mode of the data distribution. This defines the Cj. Set the parameters for prior distributions on the 6 parameters that are conjugate to the likelihoods. For the normal distribution the priors are defined in Eq. (15), so the full prior for the n components is... 

This expression is valid for a single observation y. For multiple observations, we derive p(y Q) from the fact that p(y i, O-) = Ilip yi i, G-). The result is that the likelihood is also normal with the average value of y, y, substituted for y and o -hi substituted for G in Eq. (14). The conjugate prior distribution for Eq. (14) is... 

Prior distributions are often chosen to simplify the form of the posterior distribution. The posterior density is proportional to the product of the likelihood and the prior density and so, if the prior density is chosen to have the same form as the likelihood, simplification occurs. Such a choice is referred to as the use of a conjugate prior distribution see Lee (2004) for details. In the regression model (1), the likelihood for /3, a can be written in terms of the product of a normal density on /3 and an inverse gamma density on a. This form motivates the conjugate choice of a normal-inverse-gamma prior distribution on (3, a. Additional details on this prior distribution are given by Zellner (1987). 

Let Xj denote the actual downtime associated with test j for a fixed a>. We assume that Xj N(0, cr ), when the parameters are known. Here 0 is a function of o), but cr is assumed independent of a). Hence the conditional probability density of X = (Wj,j = 1,2,..., k),p(x I jS, 0-2), where /3 = (/3q, /3i), can be determined. We would like to derive the posterior distribution for the parameters fi and the variance given observations X = x. This distribution expresses our updated belief about the parameters when new relevant data are available. To this end we first choose a suitable prior distribution for fi and a. We seek a conjugate prior which leads to the normal-inverse-gamma (NIG) distribution p(/3, o ), derived from the joint density of the inverse-gamma distributed and the normal distributed fi. The derived posterior distribution will then be of the same distribution class as the prior distribution. 

Normal distribution is conjugate to itself for fi when (see Table 1/5.3.5-1) is known. When a normal prior distribution is combined witb tbe normal likelihood function, it would lead to a normal posterior distribution. 

When the observations come from a norntal fi,a ) distribution where the variance cr is known, the conjugate prior foris normaKjn, s ). The posterior is normal m, (s ) ) where... 

When the observations come from a normal fi,conjugate prior is the product of 5 times an inverse chi-squared distribution with k degrees of freedom for times a normal m, prior for fx given cr. Note no is the equivalent sample size of the prior for fi. 

When we have n independent observations from the normal linear regression model where the observations all have the same known variance, the conjugate prior distribution for the regression coefficient vector /3 is multivariate normal(bo, Vq). The posterior distribution of /3 will be multivariate nor-mal y>i, Vi), where... 

The second approach is to use joint prior for the two parameters that is conjugate to the observations from the joint distribution where both parameters are unknown. We saw in Section 4.6 that the joint conjugate prior for a sample from a normal distribution where both the mean p and the variance cr are unknown parameters is the product of a 5 times an inverse chi-squared prior for and a normal m,... 

When we have eensored survival times data, and we relate the linear predictor to the hazard function we have the proportional hazards model. The function BayesCPH draws a random sample from the posterior distribution for the proportional hazards model. First, the function finds an approximate normal likelihood function for the proportional hazards model. The (multivariate) normal likelihood matches the mean to the maximum likelihood estimator found using iteratively reweighted least squares. Details of this are found in Myers et al. (2002) and Jennrich (1995). The covariance matrix is found that matches the curvature of the likelihood function at its maximum. The approximate normal posterior by applying the usual normal updating formulas with a normal conjugate prior. If we used this as the candidate distribution, it may be that the tails of true posterior are heavier than the candidate distribution. This would mean that the accepted values would not be a sample from the true posterior because the tails would not be adequately represented. Assuming that y is the Poisson censored response vector, time is time, and x is a vector of covariates then... 

Conjugate pair In Bayesian estimation, when the observation of new data changes only the parameters of the prior distribution and not its statistical shape (i.e., whether it is normal, beta, etc.), the prior distribution on the estimated parameter and the distribution of the quantity (from which observations are drawn) are said to form a conjugate pair. In case the likelihood and prior form a conjugate pair, the computational burden of Bayes rule is greatly reduced. 

---