Bayesian Regularisation

Bruneau, P., McElroy, N. R. Log D " modeling using Bayesian regularised neural networks. Assessment and correction of errors of prediction. 

The Bayesian regularisation algorithm updates the weight and bias values according to Levenberg-Marquardt optimisation. It minimises a combination of squared errors and weights, and then determines the correct combination so as to produce a network that generalises well. 

As the Bayesian formulation was described in Section 5 it is sufficient to recall the main uses of the formulation in the maximum a posteriori (MAP) mode or in the stochastic sampling mode. The maximum likelihood estimation method is obtained by setting the prior to unity in the MAP method. The MLE method is essentially the least squares method. Without a suitable choice of prior it may be necessary to introduce further ad hoc regularisation in the case of MLE. A carefully chosen prior should regularise the problem in a satisfactory way. 

---