Hyperparameters

The prior hyperparameters, j.oi, etc., can be estimated from the data assigned to each component. First define A = Ejli I(Cj = i), where l(Cj = i) = if Cj = i and is 0 otherwise. Then, for instance, the prior hyperparameters for the mean values are defined by... 

Being qualitative, the relative strength of the a priori with respect to the likelihood must therefore be adjustable. This can be achieved thanks to an hyperparameter p ... 

In order to finally solve our inverse problem, we have to choose an adequate level of regularization. This section presents a few methods to select the value of p. Titterington et al. (1985) have made a comparison of the results obtained from different methods for choosing the value of the hyperparameters. 

In the 4th level, prior distribntions are defined for the variance parameters and the hyperparameter ... 

High-pass filter, 156, 160 Hilbert transform definition, 409 Householder matrix, 125 Huffman coding, 65 Hybrid filter bank, 56 Hyperparameters, 182... 

The sample of individuals is assumed to represent the patient population at large, sharing the same pathophysiological and pharmacokinetic-dynamic parameter distributions. The individual parameter 0 is assumed to arise from some multivariate probability distribution 0 / (T), where jk is the vector of so-called hyperparameters or population characteristics. In the mixed-effects formulation, the collection of jk is composed of population typical values (generally the mean vector) and of population variability values (generally the variance-covariance matrix). Mean and variance characterize the location and dispersion of the probability distribution of 0 in statistical terms. 

Section 4 reviews simple, semi-automatic methods of choosing the hyperparameters of a prior distribution and adds some new insights into the choice of hyperparameters for a prior on regression coefficient vector (3. The glucose experiment and a simulated data set are used in Section 5 to demonstrate the application of the Bayesian subset selection technique. 

A Bayesian analysis proceeds by placing prior distributions on the regression coefficient vector (3, error standard deviation a, and subset indicator vector 6. One form of prior distribution is given in detail below and other approaches are then discussed. Techniques for choosing hyperparameters of prior distributions, such as the mean of a prior distribution, are discussed later in Section 4. 

A variety of prior distributions for /3 has been proposed in the literature. Here, the following formulation is used for any given subset indicator vector 6 and value of the error variance a2, the effects f>., (ih have independent normal prior distributions. The variance of Pj(j = 1,..., li) is formulated to be large or small depending on whether an effect is active (Sj = 1) or inactive (Sj = 0), through use of the hyperparameters cj, tj in the distribution with density... 

Effect sparsity and effect hierarchy are represented through the choice of hyperparameters n, jr0, tti, n2. Typically ttq < nx < n2 < n < 0.5. Section 2.3 provides details on the selection of these hyperparameters. 

Choices of values for the hyperparameters, v, A, c,-, rj, of the prior distributions (3) and (4) are now discussed, with an emphasis on automatic methods that use simple summaries of the observed data. Many of these suggestions have been made by Chipman et al. (1997) and Chipman (1998). Some suggestions relating to the choice of c and r are new. 

I propose that the hyperparameters for the prior distribution (3) on a2 are chosen so that the mean and 99th quantile of the distribution are consistent with the observed values of the response. The prior expected value of a2 is... 

Prior distribution (4) for /3 is defined by the hyperparameters Cj and rj (j = 1,. /i). In choosing these hyperparameters, it is helpful to recall from (4) that the coefficient Pj associated with an inactive contrast has standard deviation a rj and, if the contrast is instead active, then Pj has a standard deviation that is Cj times larger. As mentioned earlier in Section 2.2, intercept p0 is always present in the model, so taking r, = 1, t0 — oo gives a flat prior density for p0. 

First, a simplified method of choosing the hyperparameters is described following Bingham and Chipman (2002). The probability of an active interaction is assumed to be proportional to n, the probability of an active linear main effect, with the size of the proportionality constant dependent on which parents are active. Thus (9) becomes... 

The value of the hyperparameter jt may be chosen by considering the prior expected number of active effects. Illustrative calculations are now given for a full second-order model with / factors, and for subsets of active effects that include linear and quadratic main effects and linear x linear interactions. Thus, the full model contains / linear effects, / quadratic effects, and ( ) linear x linear interaction effects. Prior probabilities on the subsets being active have the form of (22) and (23) above. A straightforward extension of the calculations of Bingham and Chipman (2002) yields an expected number of active effects as... 

Table 7. Expected numbers of active effects, corresponding hyperparameter n, and breakdown by linear main effects and linear x linear interaction effects, for the simulated experiment with 11 factors...

The main results of the analysis of the glucose data have already been presented in Section 1.1. Details of the hyperparameter choices, robustness calculations, and estimation of the total probability visited by the MCMC sampler are given here. 

The hyperparameters in the prior distributions of a2 and /3 are set as follows for this example, unless otherwise indicated, v = 5, A. = s2/25 = 101.2282/25 = 4.049 Cj = c = 100 and tj are specified according to (21). The exact values of tj vary with effect index j, because the factor levels have different ranges. As discussed in Section 4.2, c = 10 seems to allow too much flexibility for the inactive effects to capture residual error. Calibration of jt via an expected number of effects is difficult, because effects of so many types (linear, quadratic, linear x linear, linear x quadratic, quadratic x quadratic) are present but calibration can be carried out using only the expected number of linear and quadratic main effects and linear x linear interaction effects. There are 8 possible linear, 7 quadratic, and 28 linear x linear interaction effects, for a total of 43 possible effects. The choice of jt = 0.2786 gives 5 effects expected to be active out of the 43. The inclusion of higher-order interactions will raise this expectation, but not by much, because all their parents are of at least second order and are unlikely to be active. 

In the mixed-effects context, the collection of population parameters is composed of a population-typical value (generally the mean) and of a population-variability value (generally the variance-covariance matrix). The mean and variance are the first two moments of a probability distribution. They build a minimal set of hyperparameters or population characteristics for it, which is sufficient (in a statistical sense) when F is taken as normal or log-normal. 

M. Tod, E. Mentre, Y. Merle, and A. Mallet, Robust optimal design for the estimation of hyperparameters in population pharmacokinetics. J Pharmacokinet Biopharm 26 689-716 (1998). 

Another internal validation technique is the posterior predictive check (PPC), which has been used in the Bayesian literature for years, but only recently reported in the PopPK literature by Yano, Beal, and Sheiner (2001). The basic idea is an extension of the predictive check method just described but include hyperparameters on the model parameters. Data are then simulated, some statistic of the data that is not based on the model is calculated, e.g., half-life or AUC by noncompartmental method, and then compared to the observed statistic obtained with real data. The underlying premise is that the simulated data should be similar to the observed data and that any discrepancies between the observed and simulated data are due to chance. With each simulation the statistic of interest is calculated and after all the simulations are complete, a p-value is determined by... 

We note that we may choose different values for the hyperparameter Tq/ for those location parameters including jg, jy 0, and 0g under the random effects model and yg, yy 0jt, and 0ojt under the fixed effects model. We also note that there is no information about y and 0 (or 0) in the historical data. 

---