Knowledge, statistics and thermodynamics

computation and communication deal with the processing of information. Information, on its turn, can assume different forms it can be an image, a text, a set of numbers, a sound, etc. Besides, sometimes we want information to be publicly displayed, as in an advertisement, and sometimes we want it to be secret, as when we buy something through the Internet using our credit cards. Whatever the situation, information can always be converted into a bunch of O s and 1 s. But to make something useful with information, we must have the means to represent the Os and Is in the physical world, in ordCT we can process information. In doing so, information becomes subject to the laws of physics. 

In computers, information is represented by bits in electronic circuits. But any physical object with two clearly stable distinguishable states can represent a bit of information. A collection of such objects is a physical system where information can be stored and processed. Let us take a very simple object to work as a bit a coin. The head and tail states of a regular coin are very stable, and they can be associated to the usual logic labels of a bit 0 and T . How much information can be stored in a collection of coins For the sake of argument let us considerer 4 coins. Heads will be represented by an empty circle, and tails by a full circle. Let us associate the logic label 0 to heads, and 1 to tails. 

let us imagine that instead of choosing one particular configuration among the 16 available ones, we simply adopt a statistical procedure we throw each coin and simply 

The fact that the configuration with two heads and two tails is the most likely one has nothing to do with physics it is purely statistical. It reflects the fact that if we take a single coin and throw it a large number of times, at the end we will have about half heads and half tails. 

This number somehow quantifies your lack of information before you open your eyes At the moment you open them, you become aware of the actual configuration, and all Pk collapse to zero, except a specific one, which will be equal to 1. In this situation, the entropy is S = 0. 

---