Information Theoretic Approach to Statistical Mechanics

The outcome of the reasoning hinted at above is the claim that the function H is given by 

To see the workings of the maximum entropy approach, we begin with the trivial example of N outcomes with our only knowledge being the enumeration of the 

Note that we have provided theoretical justification for the principle of equal a priori probabilities that was adopted earlier. We see that this is the best guess that can be made on our very limited data. The key point is that rather than asserting equal a priori probabilities as an article of faith we have shown that it is the best distribution that we can possibly assign in the sense that it most faithfully represents our degree of ignorance concerning the various outcomes. 

A more enlightening example is that in which not only is the constraint of normalization of the probability distribution imposed, but it is assumed that we know the average value of some quantity, for example, E) = EiPi Ei). In this case, we again consider an augmented function, which involves two Lagrange multipliers, one tied to each constraint, and given by 

Note that once again the first constraint imposes normalization of the distribution. Maximizing this augmented function leads to the conclusion that pj =.  

---

E. T Jaynes Papers on Probability and Statistics edited by R. D. Rosenkranz, D. Reidel Publishing Company, Dordrecht The Netherlands, 1983. The presence of this work on the list reflects my enormous prejudice in favor of the information theoretic approach to statistical mechanics as pioneered by Edwin T. Jaynes. This book is one of my prize possessions and at the time of this writing has no trace of its original hardback cover as a result of the many trips the book has made. 

---