Maximum Entropy Maxent and Bayesian Methods

The fundamental application of maximum entropy techniques requires an understanding of Bayes theorem. Various definitions are necessary  

The aim of the experimenter is to obtain as good an estimate of map space as possible, consistent with his or her knowledge of the system. Normally there are two types of knowledge  

Prior knowledge is available before the experiment. There is almost always some information available about chemical data. An example is that a true spectrum will always be positive we can reject statistical solutions that result in negative intensities. Sometimes much more detailed information such as lineshapes or compound concentrations is known. 

Experimental information, which refines the prior knowledge to give a posterior model of the system. 

Many scientists ignore the prior information, and for cases where data are fairly good, this can be perfectly acceptable. However, chemical data analysis is most useful where the answer is not so obvious, and the data are difficult to analyse. The Bayesian method allows prior information or measurements to be taken into account. It also allows continuing experimentation, improving a model all the time. 

---