The Information-Theoretic Approach

The Infonnation-Theoretic Approach.— An increasing number of kineticists are now tabulating their experimental results by reference to a new analytical method. This approach, which was first formulated by Bernstein and Levine, is based on concepts borrowed from information theory. One of its merits is the possiUlity of characterizing a wealth of detailed information, for example, a large set of k(v J T), in terms of a small number of parameters. Fiulhermore, the method provides a means of obtaining detailed rate data by extrapolation, where an experiment has given only partial information. 

The central feature of the information-theoretic analysis is that it proposes a quantitative measure of the surprisal of a particular outcome, for example, the observation that in reaction (20) 42% of the HCl product is formed in v = 3. To do this, it is first necessary to establish a datum line, i.e. to d ne what result would cause zero surprise. Then an equation must be formulated which determines the deviation of the actual result from the prior expectation. 

The surprisal of a particular outcome, for example, /( 0, is d ned in terms of the observed probability of the specified result, P v% and that expected on prior grounds, P°(v% according to the equation  

The surprisal is a measure of the deviation of a single population from its expected value. The two parameters A , and exp(Aa), or similar parameters for other distributions, characterize the deviation of the whole distribution from prior expectation when this can be described by equation (30). Finally, the average vaiue of this deviation can be calculated and it is known as the entropy of the distribution. The entropy deficiency 2,43,93 vibrational distribution is then d ned by 

The greater the specificity of energy disposal in an exothermic reaction, then the larger is its entropy deficiency. Values of A., and KS,. for some well-studied atom-transfer reactions are listed in Table 1 /J is the av age fraction of available energy that is released as product vibration, i.e. 

---

K. Eckschlager and V. Stepanek, Analytical Measurement and Information Advances in the Information Theoretic Approach to Chemical Analysis, Research Studies Press, Letchworth, Hertfordshire, UK, 1985. 

We now define what information each receiver must store. If we tiy and repeat the information-theoretic approach of the previous scheme where each receiver needs to store explicitly the keys of all the subsets it belongs to, the storage requirements would expand tremendously consider a receiver u for each complete subtree it belongs to, u must store a number of keys proportional to the number of nodes in the subtree Tfc that are not on the path from the root of to u. There are log N such trees, one for each height 1 < A < log N, yielding a total of — k) which is 0 N) keys. 

E. T Jaynes Papers on Probability and Statistics edited by R. D. Rosenkranz, D. Reidel Publishing Company, Dordrecht The Netherlands, 1983. The presence of this work on the list reflects my enormous prejudice in favor of the information theoretic approach to statistical mechanics as pioneered by Edwin T. Jaynes. This book is one of my prize possessions and at the time of this writing has no trace of its original hardback cover as a result of the many trips the book has made. 

We may approach the problem of latent properties the same way as one used the information-theoretical approach in the case of the Hohenberg Kohn theorem where is the information stored concerning all the latent properties of a given molecule A It is evident that many of these latent properties are eventually exhibited in response to some specific interactions, and the interacting partner or partners must have a role in triggering the manifestation of the latent properties. It is also true, however, that a different molecule B has different latent properties and the process involving the same interaction partner or partners leads to a different set of latent properties. 

The same two pieces of information is the starting point in the information-theoretical approach. For simplicity, we shall restrict the following argument to molecules however, the treatment is equally valid for any physical system built from nuclei and electrons. 

Levine, R. D. 1978. The Information Theoretic Approach to Molecular Reaction Dynamics, Annu. Rev. Phys. Chem. 29, 59. 

Already familiar a decade ago was the information-theoretic approach [15] to the characterization of disequilibrium population distributions and the applicability of surprisal analysis to non-statistical quantum state distributions and branching ratios [16]. 

With the maximum entropy principle and the notion of quasi-static processes at om disposal, we showed that the thermodynamics of a system emerges from the statistical mechanics of that system. Unlike conventional approaches to equilibriitm statistical mechanics, the information-theoretic approach does not require us to appeal to the phenomenological equations of thermodynamics in order to render the statistical mechanical formalism meaningful. 

The dynamical features associated with the light atom anomaly, e.g., direct mechanism without snarled trajectories (except for XX cases) and short reaction times compared with the vibrational periods of molecular vibrations have made the H + X2 reactions a favorite for development of models for impulsive energy release.The approximately collinear reaction configuration (at least for F2 and CI2) provide n incentive to those interested in rigorous one-dimensional quantum lodels. " The nonlinear surprisals also have been a stimulus for work based upon the information-theoretic approach. The vibrational (especially so for the onedimensional base and translationaF " surprisals are approximately quadratic, rather than linear. These distributions can be understood in terms of an additional constraint, which is termed a Franck-Condon-like or momentum transfer constraint. Introduction of this constraint leads to a... 

The recent theoretical approach based on the information theory (IT) in studying aqueous solutions and hydration phenomena [62 66] shows such a direction. IT is a part of the system based on a probabilistic way of thinking about communication, introduced in 1948 by Sharmon and subsequently developed [114]. It consists in the quantitative description of the information by defining entropy as a function of probability... 

The present theoretical approach to rubberlike elasticity is novel in that it utilizes the wealth of information which RiS theory provides on the spatial configurations of chain molecules. Specifically, Monte Carlo calculations based on the RIS approximation are used to simulate spatial configurations, and thus distribution functions for end-to-end separation r of the chains. Results are presented for polyethylene and polydimethylsiloxane chains most of which are quite short, in order to elucidate non-Gaussian effects due to limited chain extensibility. 

Qualitative molecular-orbital theory approaches (and related qualitative treatments) are discussed throughout the text (particularly in Chapters 4 and 6), and a more detailed discussion of the contributions of such approaches presented in Chapter 8. As with the experimental methods discussed in Chapter 2, the topics presented in the present chapter are associated with numerous abbreviations and acronyms (and alternative titles). Both to serve as a key to these abbreviations, and as a source of reference to the numerous theoretical approaches now available, they are listed along with brief descriptions and references to further information in Appendix C. 

Electronic structure calculations of the type described above, provide the energy and related properties of the system at the absolute zero of temperature and do not account for any time-dependent effect. In some cases, temperature and/or time scale effects may be important and must be included. The appropriate theoretical approach is then molecular dynamics (MD) either in the classical or ab initio implementations. In the first approach, Newton s motion equations are solved in the field of a potential provided externally, which constitutes the main limitation of this approach. To overcome this problem, ab initio Molecular Dynamics (AIMD)94,95 solves Newton s motion equations using the ab initio potential energy surface or propagating nuclei and electrons simultaneously as in the Car-Parrinello simulation.96 The use of AIMD simulations will increase considerably in the future. In a way they furnish all the information as in classical MD, but there are no assumptions in the way the system interacts since the potential energy surface is obtained in a rather crude manner. 

To further illustrate that information-theoretic approaches and frequentist approaches may reach different conclusions, consider the case where the full model has log-likelihood LLf and f-estimable parameters while the reduced model has log-likelihood LLr and r-estimable parameters. The likelihood ratio test is then... 

Hence, intravenous data were modeled first, followed by inhalational, then intranasal. Once the pharmacokinetics of each individual route of administration was established, all model parameters were then estimated simultaneously. Initial values for cocaine pharmacokinetics after intravenous administration were estimated using noncompartmental methods. Total systemic clearance was estimated at 100 L/h and volume of distribution at steady-state was estimated at 232 L. Central compartment clearance and intercompartmental clearance were set equal to one-half total systemic clearance (50 L/h), whereas central and peripheral compartment volumes were set equal to one-half volume of distribution (116 L). Data were weighed using a constant coefficient of variation error model based on model-predicted plasma concentrations. All models were fit using SAAM II (SAAM Institute, Seattle, WA). An Information-Theoretic approach was used for model selection, i.e., model selection was based on the AIC. 

It is the main purpose of this review to summarize all these novel developments in the combined DFT-Information-Theoretic approach to molecular and reactive systems. We shall begin, however, with a short survey of the key ideas in the theory of chemical reactivity, emphasizing a need for the truly two-reactant description. The conceptual advantages of the DFT and information theory in making a connection to the classical ideas and concepts of chemistry are elucidated throughout. 

The general result that the ground-state electron density determines the molecular properties is not really surprising if one takes an information-theoretical approach to the Hohenberg-Kohn density functional model. 

---