Ockham factor

Keywords asymptotic expansion evidence information entropy Markov Chain Monte Carlo simulation modal identification Ockham factor regression problem robustness seismic attenuation... 

In the next section, the Bayesian model class selection method is introduced for quantification and selection of model classes. It will be discussed for the globally identifiable case and the general case. The Ockham factor is introduced and it serves as the penalty for a complicated model, which appears naturally from the evidence. Computational issues will be discussed and... 

Next, it is attempted to show that the Ockham factor decreases exponentially with the number of uncertain parameters in the model class. For this purpose, consider an alternative... 

Since the prior variances will always be greater than the posterior variances if the data provide any information about the model parameters in the model class Cj, all the terms in the first summation in Equation (6.15) will be positive and so will the terms in the second summation unless the posterior most probable value 6 just happens to coincide with the prior most probable value O . Thus, one might expect that the log-Ockham factor In Oj will decrease if the number of parameters Nj for the model class Cj is increased. This expectation is confirmed by noting that the posterior variances are inversely proportional to the number of data points N in V, so the dependence of the log-Ockham factor is ... 

A comparison of this equation and Equation (6.11) shows that the Ockham factor is approximately equal to the ratio p 9 Cj)/p 9 VXj) which is always less than unity if the data provide any information about the model parameters in the model class Cj. Indeed, for large N, the negative logarithm of this ratio is an asymptotic approximation of the information about 0 provided by data V [147]. Therefore, the log-Ockham factor In Oj removes the amount of information about 9 provided by T> from the log-likelihood In p T> 9, Cj) to give the log-evidence, In p(V Cj). 

The Ockham factor can also be interpreted as a measure of robustness of a model class. If the updated PDF for the model parameters for a given model class is very peaked, then the ratio p(9 Cj)/p(9 V, Cj), and so the Ockham factor, is very small. However, a narrow peak implies that response predictions using this model class will depend too sensitively on the optimal parameters in 9. Small errors in the parameter estimation will lead to large errors in the response predictions. Therefore, a class of models with a small Ockham factor will not be robust to measurement noise during parameter estimation, i.e., during selection of the optimal model within the class. 

To summarize, in the Bayesian approach to model selection, the model classes are ranked according to p V Cj)P(Cj U) for 7 = 1,2,..., Nc, where the most plausible class of models representing the system is the one which gives the largest value of this quantity. The evidence p V Cj) can be calculated for each class of models using Equation (6.11) where the likelihood p V 9, Cj) is evaluated using the methods presented in Chapters 2-5. The prior distribution P Cj U) over all the model classes Cj, j = 1,2,..., Nc, can be used for other concerns, such as computational demand. However, it is out of the scope of this book and uniform prior plausibilities are chosen, leaving the Ockham factor alone to penalize the model classes. 

In practice, the likelihood value, Ockham factor and evidence may associate with a large order for large N. Computational problems (i.e., giving either zero or infinity) may be encountered for direct calculation and/or normalization of the plausibilities. To resolve this problem, one first calculates the log-likelihood and the log-Ockham factor and hence the log-evidence, denoted by In p(I> Ci), In p(I> C2),. ..,ln p(V Cnc)- Instead of taking the exponential of the log-evidence and then normalizing the plausibility, the maximum log-evidence is subtracted from the log-evidence of each model class and then taking the exponential of this array. This operation does not affect the relative plausibility between different model classes. Finally, the plausibility of a model class can be obtained by normalizing this array ... 

By using Equation (6.14) with Equation (6.48), the Ockham factor is given by ... 

Independent prior distributions for the parameters are taken as follows a Gaussian distribution for the natural frequencies with mean 0.5(2m - 1) Hz and a coefficient of variation 10% for the mth mode. Furthermore, the Rayleigh coefficients, the modal participation factor and the spectral intensity of the prediction error are assumed to be uniformly distributed over a sufficiently wide range to let the likelihood function determine their values. Note that the ranges of these distributions do not affect the model class selection results since they influence all modal models in the same way. Therefore, the computation of the Ockham factor and plausibility will exclude the prior PDF of these parameters but the one for the modal frequencies will still be included. 

---