Average amount of information before reporting the result

5 Average amount of information before reporting the result 

In the previous section, the amount of information that is obtained by reporting the result was shown. In this section, the amount of uncertainty about the result obtained before reporting the result is shown. In other words, a method of expressing the amount of uncertainty of the result reported is clarified. In this section, in order to make the discussion more concrete, the following uncertainties are treated  

Before the result is known, the amount of uncertainty of the result can be expressed as the average of the respective amounts of information obtained when each of the results is reported. The average amount of information is obtained as the sum of the product of the amount of information and the probability of occurrence for each result as 

This average amount of information is called as information entropy. In particular, when the system is a single event/phenomena, the information entropy is called self-entropy. For example, the self-entropy in the case of casting an unbiased die becomes 

This value is identical to that for the case where the news number five occurs is given, as was described in the previous subsection. In the case of casting a biased die, which has a probability that number one occurs as 1/2 and the probability that other numbers occur as 1/10, the self-entropy becomes 

---