Thermodynamics and Information Theory

Any computation, whether performed by a slide rule, computer work station or brain, is inherently a physical process, and as such is therefore subject to whatever laws and limitations apply to physical systems in general. It is a natural question to ask, then, whether there exists a fundamental therniodynamic limit to computation i.e, whether there is a rniiiimuin amount of energy that is required to perform a given logical operation  

There are three important milestones in the history of tliermodynamics and computation  

The earliest hint that physics and information might be more than just casually related actually dates back at least as far as 1871 and the publication of James Clerk Maxwell s Theory of Heat, in which Maxwell introduced what has become known as the paradox of Maxwell s Demon. Maxwell postulated the existence of a hypothetical demon that positions himself by a hole separating two vessels, say A and B. While the vessels start out being at the same temperature, the demon selectively opens the hole only to either pass faster molecules from A to B or to pass slower molecules from B to A. Since this results in a systematic increase in B s temperature and a lowering of A s, it appears as though Maxwell s demon s actions violate the second law of thermodynamics the total entropy of any physical system can only increase, or, for totally reversible processes, remain the same it can never decrease. Maxwell was thus the first to recognize a connection between the thermodynamical properties of a gas (temperature, entropy, etc.) and the statistical properties of its constituent molecules. 

The relationships between thermodynamic entropy and Shannon s information-theoretic entropy and between physics and computation have been explored and hotly debated ever since. It is now well known, for example, that computers can, in principle, provide an arbitrary amount of reliable computation per kT of dissipated energy ([benu73], [fredkin82] see also the discussion in section 6.4). Whether a dissipationless computer can be built in practice, remains an open problem. We must also remember that computers are themselves physical (and therefore, ultimately, quantum) devices, so that any exploration of the limitations of computation will be inextricably linked with the fundamental limitations imposed by the laws of physics. 

---

Landauer s principle The principle put forward by Rolf Landauer in the 1960s that energy has to be expended to erase information. This principle links thermodynamics and information theory. 

Chemical Thermodynamics and Information Theory with Applications... 

---