Analysis of Chemical Information Content Using Shannon Entropy

Analysis of Chemical Information Content Using Shannon Entropy 

Department of Life Science Informatics, B-IT, Rheinische Friedrich-Wilhelms-Universitdt, Bonn, Germany 

Entropy is well known as a quantitative measure of the disorder of a closed system in thermodynamics and statistical mechanics. The equilibrium of a thermodynamic system is associated with the distribution of objects or molecules having the greatest probability of occurring, and this most probable state is the one with the greatest degree of disorder. In statistical mechanics, the increase in entropy to its maximum at equilibrium is rationalized as the 

Reviews in Computational Chemistry, Volume 23 edited by Kenny B. Lipkowitz and Thomas R. Cundari Copyright 2007 Wiley-VCH, John Wiley Sons, Inc. 

Claude E. Shannon is generally recognized as the founding father of information theory as we understand it today a mathematical theory or framework to quantitatively describe the communication of data. Irrespective of their nature or type, data need to be transmitted over channels, and a focal point of Shannon s pioneering work has been that channels available for communicating data are generally noisy. Shannon demonstrated that data can be communicated over noisy channels with a small probability of error if it is possible to encode (and subsequently) decode the data in a way that communicates data at a rate below but close to channel capacity. 

---

Jeffrey W. Godden and Jurgen Bajorath, Analysis of Chemical Information Content Using Shannon Entropy. 

We will describe below the SE formalism in detail and explain how it can be used to estimate chemical information content based on histogram representations of feature value distributions. Examples from our work and studies by others will be used to illustrate key aspects of chemical information content analysis. Although we focus on the Shannon entropy concept, other measures of information content will also be discussed, albeit briefly. We will also explain why it has been useful to extend the Shannon entropy concept by introducing differential Shannon entropy (DSE) to facilitate large-scale analysis and comparison of chemical features. The DSE formalism has ultimately led to the introduction of the SE-DSE metric. 

---