Prior hyperparameters

The prior hyperparameters, j.oi, etc., can be estimated from the data assigned to each component. First define A = Ejli I(Cj = i), where l(Cj = i) = if Cj = i and is 0 otherwise. Then, for instance, the prior hyperparameters for the mean values are defined by... 

Substitute test data of all process into Equations (3), (7) and (8) to obtain prior hyperparameters a = 0.4045, b = 1974.5, then combine all the information above to derive inheritance factor. By calculating Equations (9) and (10), we can get the conclusion that the reliability of the product do not satisfy the requirement it is needed to continue the reliability qudification test. 

In the 4th level, prior distribntions are defined for the variance parameters and the hyperparameter ... 

Section 4 reviews simple, semi-automatic methods of choosing the hyperparameters of a prior distribution and adds some new insights into the choice of hyperparameters for a prior on regression coefficient vector (3. The glucose experiment and a simulated data set are used in Section 5 to demonstrate the application of the Bayesian subset selection technique. 

A Bayesian analysis proceeds by placing prior distributions on the regression coefficient vector (3, error standard deviation a, and subset indicator vector 6. One form of prior distribution is given in detail below and other approaches are then discussed. Techniques for choosing hyperparameters of prior distributions, such as the mean of a prior distribution, are discussed later in Section 4. 

A variety of prior distributions for /3 has been proposed in the literature. Here, the following formulation is used for any given subset indicator vector 6 and value of the error variance a2, the effects f>., (ih have independent normal prior distributions. The variance of Pj(j = 1,..., li) is formulated to be large or small depending on whether an effect is active (Sj = 1) or inactive (Sj = 0), through use of the hyperparameters cj, tj in the distribution with density... 

Choices of values for the hyperparameters, v, A, c,-, rj, of the prior distributions (3) and (4) are now discussed, with an emphasis on automatic methods that use simple summaries of the observed data. Many of these suggestions have been made by Chipman et al. (1997) and Chipman (1998). Some suggestions relating to the choice of c and r are new. 

I propose that the hyperparameters for the prior distribution (3) on a2 are chosen so that the mean and 99th quantile of the distribution are consistent with the observed values of the response. The prior expected value of a2 is... 

Prior distribution (4) for /3 is defined by the hyperparameters Cj and rj (j = 1,. /i). In choosing these hyperparameters, it is helpful to recall from (4) that the coefficient Pj associated with an inactive contrast has standard deviation a rj and, if the contrast is instead active, then Pj has a standard deviation that is Cj times larger. As mentioned earlier in Section 2.2, intercept p0 is always present in the model, so taking r, = 1, t0 — oo gives a flat prior density for p0. 

The value of the hyperparameter jt may be chosen by considering the prior expected number of active effects. Illustrative calculations are now given for a full second-order model with / factors, and for subsets of active effects that include linear and quadratic main effects and linear x linear interactions. Thus, the full model contains / linear effects, / quadratic effects, and ( ) linear x linear interaction effects. Prior probabilities on the subsets being active have the form of (22) and (23) above. A straightforward extension of the calculations of Bingham and Chipman (2002) yields an expected number of active effects as... 

The hyperparameters in the prior distributions of a2 and /3 are set as follows for this example, unless otherwise indicated, v = 5, A. = s2/25 = 101.2282/25 = 4.049 Cj = c = 100 and tj are specified according to (21). The exact values of tj vary with effect index j, because the factor levels have different ranges. As discussed in Section 4.2, c = 10 seems to allow too much flexibility for the inactive effects to capture residual error. Calibration of jt via an expected number of effects is difficult, because effects of so many types (linear, quadratic, linear x linear, linear x quadratic, quadratic x quadratic) are present but calibration can be carried out using only the expected number of linear and quadratic main effects and linear x linear interaction effects. There are 8 possible linear, 7 quadratic, and 28 linear x linear interaction effects, for a total of 43 possible effects. The choice of jt = 0.2786 gives 5 effects expected to be active out of the 43. The inclusion of higher-order interactions will raise this expectation, but not by much, because all their parents are of at least second order and are unlikely to be active. 

The choice of hyperparameters <5, 0 and cr depends strongly on the data set. 0 represents the width of the basis functions, i.e. it characterises the typical length scale over which the function values become uncorrelated. <3 places a prior on the variance of the parameter vector w, describing the typical variance of the function, while is the assumed noise in the measured data values. Ideally, a prediction for fjv+i would be made by evaluating the integral... 

The joint distribution of all the parameters and the data can be easily found using the dependence structure shown on the graph. First start with the prior distributions of the hyperparameters at the top of the hierarchy. Then multiply by the conditional distribution of each of their child nodes given the parent and copareni nodes. Continue this working down the hierarchy of the parameters. Lastly multiply the conditional distribution of each the data value, given its parent nodes. The joint distribution of... 

In the hierarchical model, the parameters of the parameter distribution are known as hyperparameters. We don t really have a great deal of prior information about the hyperparameters. We might think that improper priors would be appropriate. Unexpectedly however, this approach can lead to trouble. 

Example 17 We have ten observations from each of three groups, and two predictor variables recorded for each observation. The data are given in Table 10.3. We set the priors for the hyperparameters. We chose a normal(0,1000 ) prior for t, a 500 times an inverse chi-squared prior for ip and for o, and a bivariate normal(bo, B ) prior for the vector /3 where... 

We show that using an improper Jeffrey s prior distribution for the hypervariance can lead to an improper joint posterior for all the parameters in a hierarchical model. This is despite the Gibbs sampling conditional distribution for each node being proper We will show that using a Jeffrey s prior for a scale hyperparameter causes the joint posterior to be improper since the data is not able to drive the posterior away from the vertical asymptote at zero. 

Suppose we have a hierarchical mean model. We will draw a single independent observation from normal distributions having different means but equal variance. The observation yj comes from a normal iij,a ) distribution where we may assume the variance is known. Each of the means /ij is an independent draw from a normal T, ip) distribution for j = 1,..., J. This is shown in Figure A.l. To do a full Bayesian analysis we will have to put priors on the hyperparameters. Suppose we decide to use Jeffrey s prior for the variance and an independent flat prior for the hypermean r. If we look at each step of the Gibbs sampler, everything looks ok. The... 

---