Information theory approximations

Heat Capacity of Solvation from Computer Simulations and from an Information Theory Approximation. 

Fig. 4. A schematic two-dimensional illustration of the idea for an information theory model of hydrophobic hydration. Direct insertion of a solute of substantial size (the larger circle) will be impractical. For smaller solutes (the smaller circles) the situation is tractable a successful insertion is found, for example, in the upper panel on the right. For either the small or the large solute, statistical information can be collected that leads to reasonable but approximate models of the hydration free energy, Eq. (7). An important issue is that the solvent configurations (here, the point sets) are supplied by simulation or X-ray or neutron scattering experiments. Therefore, solvent structural assumptions can be avoided to some degree. The point set for the upper panel is obtained by pseudo-random-number generation so the correct inference would be of a Poisson distribution of points and = kTpv where v is the van der Waals volume of the solute. Quasi-random series were used for the bottom panel so those inferences should be different. See Pratt et al. (1999).

To find /[p, u, ] for any given p, t/ , E, (functions of (x) or of (t, x)) we must find the corresponding canonical-like S. In the vestigial force case, pEare related to S only through/(or F), since inter-molecular dependences due to intermolecular potentials are absent at this level of approximation. It is an easy exercise in information theory to show that of all S having the same /, the product... 

The information theory approach to calculating approximate probabilities is quite general and, as we have just shown, is quite straight forward to use. One might then ask why we did not use this approach in the previous section to predict e2(0> e4(7),. .., e2J(t), and e4J(f) from and Aj(t) The answer to this question is that we did. That is, information theory predicts Gaussian transition probabilities for V and J and these were the transition probabilities that we assumed. We shall now elaborate on this remark. Let P(V, t V0,0) be the joint probability that a molecule has a velocity V at time t and a velocity V0 at t = 0. then P is related to the transition probability Pv by... 

The solution of a protein crystal structure can still be a lengthy process, even when crystals are available, because of the phase problem. In contrast, small molecule (< 100 atoms) structures can be solved routinely by direct methods. In the early fifties it was shown that certain mathematical relationships exist between the phases and the amplitudes of the structure factors if it is assumed that the electron density is positive and atoms are resolved [255]. These mathematical methods have been developed [256,257] so that it is possible to solve a small molecule structure directly from the intensity data [258]. For example, the crystal structure of gramicidin S [259] (a cyclic polypeptide of 10 amino acids, 92 atoms) has been solved using the computer programme MULTAN. Traditional direct methods are not applicable to protein structures, partly because the diffraction data seldom extend to atomic resolution. Recently, a new method derived from information theory and based on the maximum entropy (minimum information) principle has been developed. In the immediate future the application will require an approximate starting phase set. However, the method has the potential for an ab initio structure determination from the measured intensities and a very small sub-set of starting phases, once the formidable problems in providing numerical methods for the solution of the fundamental equations have been solved. 

As pointed out by Shannon [15], who established the information theory as an autonomous mathematical discipline, the basic problem of communication is that of reproducing at one point (receiver, output), exactly or approximately, a message sent at another point (source, input). The free (isolated) constituent atoms, defining the promolecule , can be viewed as the molecular message source. The information contained in the probability distributions of this reference state is mostly preserved in the molecule, the molecular message receiver. Indeed, the bonded (chemical) AIM are known to be only slightly perturbed in their valence shell relative to their free analogs. However, these small deformations in the electron distribution, due to... 

Other calculations tested using this molecule include two-dimensional, fully numerical solutions of the molecular Dirac equation and LCAO Hartree-Fock-Slater wave functions [6,7] local density approximations to the moment of momentum with Hartree-Fock-Roothaan wave functions [8] and the effect on bond formation in momentum space [9]. Also available are the effects of information theory basis set quality on LCAO-SCF-MO calculations [10,11] density function theory applied to Hartree-Fock wave functions [11] higher-order energies in... 

Powles has introduced an approximation based on information theory (19). If one knows the second and fourth moments of a distribution then information theory can be used to find a "best approximation to the moment generating function. Applying this to C Ct) is not completely straightforward, but Powles obtains an expression for the foiorier transform of Cj it)... 

Results from simulations show that the next step is not easy. Compressed gases and low torque liquids may reasonably be modelled by Steele s torque approximation Powles information theory expression or by the J diffusion models, although the evidence suggests that the Fokker-Planck model is not very successful. But commonly occurring high torque liquids cannot be successfully modelled by these techniques and there seems to be no simple alternative. It is likely that at least two parameters will be needed even for spherical and linear molecules. Further work to compare two parameter approximations with simulation results will show the most satisfactory next approximation. 

We now come to the question whether simulation calculations will always be required to obtain information on po-We discuss a recently proposed information theory that predicts po approximately. The idea is to consider po as the... 

Analytical information taken from a chromatogram has almost exclusively involved either retention data (retention times, capacity factors, etc.) for peak identification or peak heights and peak areas for quantitative assessment. The width of the peak has been rarely used for analytical purposes, except occasionally to obtain approximate values for peak areas. Nevertheless, as seen from the Rate Theory, the peak width is inversely proportional to the solute diffusivity which, in turn, is a function of the solute molecular weight. It follows that for high molecular weight materials, particularly those that cannot be volatalized in the ionization source of a mass spectrometer, peak width measurement offers an approximate source of molecular weight data for very intractable solutes. 

Most of these approximations are mainly of a computational nature there are well-defined methods available for going beyond the approximations, but they are computationally too demanding. The key is therefore to be able to evaluate what level of theory (i.e. which approximations are appropriate) is required to obtain results which are sufficiently accurate to provide useful information about the question at hand. 

In the next section we describe a very simple model, which we shall term the crystalline model , which is taken to represent the real, complicated crystal. Some additional, more physical, properties are included in the later calculations of the well-established theories (see Sect. 3.6 and 3.7.2), however, they are treated as perturbations about this basic model, and depend upon its being a good first approximation. Then, Sect. 2.1 deals with the information which one would hope to obtain from equilibrium crystals — this includes bulk and surface properties and their relationship to a crystal s melting temperature. Even here, using only thermodynamic arguments, there is no common line of approach to the interpretation of the data, yet this fundamental problem does not appear to have received the attention it warrants. The concluding section of this chapter summarizes and contrasts some further assumptions made about the model, which then lead to the various growth theories. The details of the way in which these assumptions are applied will be dealt with in Sects. 3 and 4. 

Once a general conformation type or preliminary classification has been established it is possible to use sedimentation data to obtain more detailed information about polysaccharide conformation. For example, the low value of ks/[v 0 25 found for the bacterial polysaccharide xylinan has been considered to be due to asymmetry [115]. If we then assume a rigid structure the approximate theory of Rowe [36,37] can be applied in terms of a prolate ellipsoid of revolution to estimate the aspect ratio p L/d for a rod, where L is the rod length and d is its diameter) 80. 

---