Statistical thermodynamics information theory

Chemical Thermodynamics Information Theory Mechanics, Classical Quantum Mechanics Statistics, Foundations Thermodynamics... 

A third hint of a connection between physics and information theory comes from the thermodynamics of Black Holes, which is still a deep mystery that embodies principles from quantum mechanics, statistical mechanics and general relativity. 

It is most remarkable that the entropy production in a nonequilibrium steady state is directly related to the time asymmetry in the dynamical randomness of nonequilibrium fluctuations. The entropy production turns out to be the difference in the amounts of temporal disorder between the backward and forward paths or histories. In nonequilibrium steady states, the temporal disorder of the time reversals is larger than the temporal disorder h of the paths themselves. This is expressed by the principle of temporal ordering, according to which the typical paths are more ordered than their corresponding time reversals in nonequilibrium steady states. This principle is proved with nonequilibrium statistical mechanics and is a corollary of the second law of thermodynamics. Temporal ordering is possible out of equilibrium because of the increase of spatial disorder. There is thus no contradiction with Boltzmann s interpretation of the second law. Contrary to Boltzmann s interpretation, which deals with disorder in space at a fixed time, the principle of temporal ordering is concerned by order or disorder along the time axis, in the sequence of pictures of the nonequilibrium process filmed as a movie. The emphasis of the dynamical aspects is a recent trend that finds its roots in Shannon s information theory and modem dynamical systems theory. This can explain why we had to wait the last decade before these dynamical aspects of the second law were discovered. 

The maximum entropy method (MEM) is an information-theory-based technique that was first developed in the field of radioastronomy to enhance the information obtained from noisy data (Gull and Daniell 1978). The theory is based on the same equations that are the foundation of statistical thermodynamics. Both the statistical entropy and the information entropy deal with the most probable distribution. In the case of statistical thermodynamics, this is the distribution of the particles over position and momentum space ( phase space ), while in the case of information theory, the distribution of numerical quantities over the ensemble of pixels is considered. 

A detailed discussion of the statistical thermodynamic aspects of thermally stimulated dielectric relaxation is not provided here. It should suffice to state that kinetics of most of the processes are again complicated and that the phenomenological kinetic theories used to described thermally stimulated currents make use of assumptions that, being necessary to simplify the formalism, may not always be justified. Just as in the general case, TSL and TSC, the spectroscopic information may in principle be available from the measurement of thermally stimulated depolarization current (TSDC). However, it is frequently impossible to extract it unambiguously from such experiments. 

The unique features of our system enable us to use three different theoretical tools — a molecular dynamics simulation, models which focus on the repulsion between atoms and a statistical approach, based on an information theory analysis. What enables us to use a thermodynamic-like language under the seemingly extreme nonequilibrium conditions are the high density, very high energy density and the hard sphere character of the atom-atom collisions, that contribute to an unusually rapid thermalization. These conditions lead to short-range repulsive interactions and therefore enable us to use the kinematic point of view in a useful way. 

In Chapter 2, we developed statistical thermodynamics as the central theory that enables ns in principle to calculate thermophysical properties of macroscopic confined flriids. A key feature of statistical thermodynamics is an enormous reduction of information that takes place as one goes from the microscopic world of electrons, photons, atoms, or molecules to the macroscopic world at which one performs measurements of thermophysical properties of interest. This information reduction is effected by statistical concepts such as the most probable distribution of quantum states (see Section 2.2.1). 

Since this expression is similar to that found for entropy in statistical mechanics, it is called the entropy of the probability distribution p,. Jaynes [260] shows that the thermodynamic entropy is identical to the information theory entropy except for the presence of the Boltzman constant in the former, made necessary by our arbitrary temperature scale. 

Thus, statistical mechanics development of the thermodynamic laws is interesting and straightforward enough, and it sheds helpful light on thermodynamics. So is the use of information theory to develop and discuss statistical mechanics. They supplement the understanding generated by the macroscopic statements, but in no sense do they replace that understanding. 

The dominant view currently held about the physical significance of thermodynamics is based on the interpretation of a "thermodynamic state" as a composite that best describes the knowledge of an observer possessing only partial information about the "actual state" of the system. The "actual state" at any instant of time is defined as a wave function (a pure state or a projection operator) of quantum mechanics. The theories that have recently evolved pursuant to this view have been called informational, though the same concept is the foundation of all statistical thermodynamics. 

At the same time transition state theory requires information about the activated complexes, assumes equilibrium only for reactants, but not products and requires introduction of a special partition function (minus one degree of freedom). Another question which remains is the applicability of statistical thermodynamics, if the life time of activated complexes is ca. 10 13 s. For instance the application of transmission coefficient contradicts the basic principles of TST, namely statistical equilibrium between reactants and activated complexes. 

Abstract Fluctuation Theory of Solutions or Fluctuation Solution Theory (FST) combines aspects of statistical mechanics and solution thermodynamics, with an emphasis on the grand canonical ensemble of the former. To understand the most common applications of FST one needs to relate fluctuations observed for a grand canonical system, on which FST is based, to properties of an isothermal-isobaric system, which is the most common type of system studied experimentally. Alternatively, one can invert the whole process to provide experimental information concerning particle number (density) fluctuations, or the local composition, from the available thermodynamic data. In this chapter, we provide the basic background material required to formulate and apply FST to a variety of applications. The major aims of this section are (i) to provide a brief introduction or recap of the relevant thermodynamics and statistical thermodynamics behind the formulation and primary uses of the Fluctuation Theory of Solutions (ii) to establish a consistent notation which helps to emphasize the similarities between apparently different applications of FST and (iii) to provide the working expressions for some of the potential applications of FST. 

Extensions of this statistical thermodynamical approach to estimating reaction rates include the RRK and RRKM theories of unimolecular decay rates, and the information theoretic formulation of reaction dynamics. These theories are remarkably successful, although generally more successful at interpreting experimental data and correlating results than at deriving results a priori. 

The intersection of the microscopic scale with information presents a vast literature. To list a sampling most helpful to the author, one begins with the information theory and statistical thermodynamics work of Jaynes [4], and the later text by Baierlein on atoms and information [5]. At a less advanced but still highly illuminating level are books by Morowitz [6,7]. Information casts a wide net in chemistry. Levine and coworkers have long championed information theory applied to molecular processes such as relaxation and internal energy redistribution [8,9]. Biopolymers plus information yield the field of bioinformatics. Recommended is the text by Tramontano for the landmaik questions posed [10]. The research of Schneider has addressed in depth the information attributes of biopolymers [11,12]. 

Using examples from physical and organic chemistry, this book demonstrates how the disciplines of thermodynamics and information theory are intertwined. Accessible to curiosity-driven chemists with knowledge of basic calculus, probability, and statistics, the book provides a fresh perspective on time-honored subjects such as state transformations, heat and work exchanges, and chemical reactions. 

The starting point of molecular simulation methods is - as in the density functional theory - the well-defined microscopic description of the system studied. This macroscopic (molecular) specification includes (1) the equations of statistical thermodynamics describing the fluid/fluid and solid/fluid interactions, and (2) the molecular model of solid adsorbent. This model should take into account all possible and reliable information on the solids, most of which can be developed from various modern surface science techniques [417]. For instance, some important data on the bulk crystalline structures are given by the X-ray diffraction or neutron diffraction, but the scanning tunelling microscopy is a valuable source of information on the topography of a surface solid. For solving... 

In order that the reader may appreciate the nature of thermodynamic quantities, thermodynamic data, and their uses, it is necessary for him to know the present state of thermodynamic theory and be aware of sources of information on the fundamental background of classical and statistical thermodynamics. This Chapter does not aim to give a short course in thermodynamics nor does it review in detail the history of the foundation of the subject with which such names as Joseph Black (1728—-99), Count Rumford (1753—1814), Sadi Carnot (1796—1832), James Joule (1818—89), and Lord Kelvin (1824—1907) are associated. 

Chapter 4 discusses fundamental questions of the validity of chemical information obtained one atom-at-a-time. While stiU presenting concepts of statistical thermodynamics and fluctuation theory, and discussing limitations of atom-at-a-time chemistry, the revised version of this chapter includes a discussion of atom-at-a-time chemistry in more general terms. 

Density functional theory from statistical mechanics is a means to describe the thermodynamics of the solid phase with information about the fluid [17-19]. In density functional theory, one makes an ansatz about the structure of the solid, usually describing the particle positions by Gaussian distributions around their lattice sites. The free... 

The earliest hint that physics and information might be more than just casually related actually dates back at least as far as 1871 and the publication of James Clerk Maxwell s Theory of Heat, in which Maxwell introduced what has become known as the paradox of Maxwell s Demon. Maxwell postulated the existence of a hypothetical demon that positions himself by a hole separating two vessels, say A and B. While the vessels start out being at the same temperature, the demon selectively opens the hole only to either pass faster molecules from A to B or to pass slower molecules from B to A. Since this results in a systematic increase in B s temperature and a lowering of A s, it appears as though Maxwell s demon s actions violate the second law of thermodynamics the total entropy of any physical system can only increase, or, for totally reversible processes, remain the same it can never decrease. Maxwell was thus the first to recognize a connection between the thermodynamical properties of a gas (temperature, entropy, etc.) and the statistical properties of its constituent molecules. 

---