Shannon’s formula

Incidentally, Equation (1.15) is also called the Shannon formula for entropy. Claude Shannon was an engineer who developed his definition of entropy, sometimes called information entropy, as a measure of the level of uncertainty of a random variable. Shannon s formula is central in the discipline of information theory. 

Suppose a certain system comprises N elements. Then its information content is determined by Shannon s formula ... 

Pielou EC (1966) Shannon s formula as a measure of species diversity its use and misuse. Amer Nature 100 463 65... 

Boltzmann s tombstone in Vienna bears the famous formula 5 = k log W (W = Wahrscheinlichkeit—probability) that was a signature of his audacious concepts. The alternative formula (13.69) (which reduces to Boltzmann s in the limit of equal a priori probabilities pa) was ultimately developed by Gibbs, Shannon, and others in a more general and productive way (see Sidebar 13.4), but the key step of employing probability to trump Newtonian determinism was his. Boltzmann was long identified with efforts to establish the //-theorem and Boltzmann equation within the context of classical mechanics, but each such effort to justify the second law (or existence of atoms) in the strict framework of Newtonian dynamics proved futile. Boltzmann s deep intuition to elevate probability to a primary physical principle therefore played a key role in efforts to find improved foundation for atomic and molecular concepts in the pre-quantum era. 

Champoux, M., Hibbeln, J.R., Shannon, C., Majchrzak, S., Suomi, S.J., Salem, N., Jr., and Higley, J.D. (2002) Fatty Acid Formula Supplementation and Neuromotor Development in Rhesus MoiJcey Neonates, Pediatr. Res. 51,273-281. 

Ethane and propane are restricted to C-H and C-C units. The same units are carried by ethanethiol, in addition to C-S and S-H. To quantify information in the first order—the bits per single ABA encounter—one tabulates the occurrence frequencies (/j) for each unit recorded on the tape. A given molecule poses Af different units. The first-order Shannon information (/j) follows from the (now) familiar formula ... 

---