Shannon-Weaver entropy

The maximum-entropy (maxEnt) approach involves the use of a measure of the uncertainty in a distribution (Shannon-Weaver entropy). The idea is to choose the distribution type that has maximum uncertainty subject to specification of some features of the distribution such as the range or a few moments or percentiles. Warren-Hicks and Moore (1998) list maxEnt solutions for a number of situations. In particular when only a min and max is available the maxEnt solution is the uniform distribution. The solution when the information available is the mean and variance, and the min and max are infinite, is the normal distribution. 

The mean information content 7, also called Shannon s entropy H [Shannon and Weaver, 1949], is defined as ... 

As we have previously seen (see Sections 2.5, 2.6, and 3.11), the maintenance of an ordered state requires energy expenditure, energy extracted from the environment. Entropy has previously been introduced (Section 2.5) as a concept of disorder. Thus, information storage (as an ordered state) and thermodynamic entropy (as a measure of disorder) are somehow related inversely. Shannon s definition of information is (Shannon and Weaver, 1949 Gatlin, 1972 Loewenstein, 1999) ... 

---