Entropy maximum

In physical chemistry, entropy has been introduced as a measure of disorder or lack of structure. For instance the entropy of a solid is lower than for a fluid, because the molecules are more ordered in a solid than in a fluid. In terms of probability it means also that in solids the probability distribution of finding a molecule at a given position is narrower than for fluids. This illustrates that entropy has to do with probability distributions and thus with uncertainty. One of the earliest definitions of entropy is the Shannon entropy which is equivalent to the definition of Shannon s uncertainty (see Chapter 18). By way of illustration we 

According to Shannon, the uncertainty of the two methods can be expressed by means of  

Shannon s uncertainty for two probability distributions (a broad and narrow distribution) 

Application of this equation to the probability distributions given in Table 40.6 shows that H for the less precise method is larger than for the more precise method. Uniform distributions represent the highest form of uncertainty and disorder. Therefore, they have the largest entropy. 

We now apply the same principle to calculate the entropy of a spectrum (or any other signal). The entropy, 5 of a spectrum given by the vector y is defined as 

---

The classical computer tomography (CT), including the medical one, has already been demonstrated its efficiency in many practical applications. At the same time, the request of the all-round survey of the object, which is usually unattainable, makes it important to find alternative approaches with less rigid restrictions to the number of projections and accessible views for observation. In the last time, it was understood that one effective way to withstand the extreme lack of data is to introduce a priori knowledge based upon classical inverse theory (including Maximum Entropy Method (MEM)) of the solution of ill-posed problems [1-6]. As shown in [6] for objects with binary structure, the necessary number of projections to get the quality of image restoration compared to that of CT using multistep reconstruction (MSR) method did not exceed seven and eould be reduced even further. 

In the Maximum Entropy Method (MEM) which proceeds the maximization of the conditional probability P(fl p ) (6) yielding the most probable solution, the probability P(p) introducing the a priory knowledge is issued from so called ergodic situations in many applications for image restoration [1]. That means, that the a priori probabilities of all microscopic configurations p are all the same. It yields to the well known form of the functional 5(/2 ) [9] ... 

W. Von der Linden Maximum-entropy data analysis. J. Appl. Physics. A. NA60, 1995, pp. 155-165. 

Remade F and Levine R D 1993 Time domain information from resonant Raman excitation profiles a direct inversion by maximum entropy J. Chem. Phys. 99 4908-25... 

A second type of relaxation mechanism, the spin-spm relaxation, will cause a decay of the phase coherence of the spin motion introduced by the coherent excitation of tire spins by the MW radiation. The mechanism involves slight perturbations of the Lannor frequency by stochastically fluctuating magnetic dipoles, for example those arising from nearby magnetic nuclei. Due to the randomization of spin directions and the concomitant loss of phase coherence, the spin system approaches a state of maximum entropy. The spin-spin relaxation disturbing the phase coherence is characterized by T. ... 

This can be inserted in equation (02.2.3) to give tlie orientational distribution function, and tlius into equation (02.2.6) to deteniiine the orientational order parameters. These are deteniiined self-consistently by variation of tlie interaction strength iin equation (c2.2.7). As pointed out by de Gemies and Frost [20] it is possible to obtain tlie Maier-Saupe potential from a simple variational, maximum entropy metliod based on tlie lowest-order anisotropic distribution function consistent witli a nematic phase. 

Uncertainly estimates are made for the total CDF by assigning probability distributions to basic events and propagating the distributions through a simplified model. Uncertainties are assumed to be either log-normal or "maximum entropy" distributions. Chi-squared confidence interval tests are used at 50% and 95% of these distributions. The simplified CDF model includes the dominant cutsets from all five contributing classes of accidents, and is within 97% of the CDF calculated with the full Level 1 model. 

A successful method to obtain dynamical information from computer simulations of quantum systems has recently been proposed by Gubernatis and coworkers [167-169]. It uses concepts from probability theory and Bayesian logic to solve the analytic continuation problem in order to obtain real-time dynamical information from imaginary-time computer simulation data. The method has become known under the name maximum entropy (MaxEnt), and has a wide range of applications in other fields apart from physics. Here we review some of the main ideas of this method and an application [175] to the model fluid described in the previous section. 

We briefly repeat now the essential parts of the maximum entropy method for details we refer to the literature [167-169]. We seek to obtain information on the dynamics of the internal degree of freedom of the model from PIMC simulations. The solution of this problem is not... 

It will now be shown from purely thermodynamic considerations that for, adiabatic conditions, supersonic flow cannot develop in a pipe of constant cross-sectional area because the fluid is in a condition of maximum entropy when flowing at the sonic velocity. The condition of the gas at any point in the pipe where the pressure is P is given by the equations ... 

The condition of maximum entropy occurs when dS/dT = 0, where ... 

The well-known maximum entropy method (MEM) can be implemented thanks to a non-quadratic regularization term which is the so-called negen-tropy ... 

The second law also describes the equilibrium state of a system as one of maximum entropy and minimum free energy. For a system at constant temperature and pressure the equilibrium condition requires that the change in free energy is zero ... 

Keck, J. C. (1978). Rate-controlled constrained equilibrium method for treating reactions in complex systems. In Maximum Entropy Formalism" (R. D. Levine and M. Tribus, eds). M.I.T. Press, Cambridge, MA. 

Table 1 Spectral analysis for Nino 3.4 index, streamflow, and AF using the Maximum Entropy Method. Only results for the low frequency band are showed ...

We started by analyzing the dominant modes of oscillation showed by oxygen content, streamflow, and ENSO using the Maximum Entropy Method (MEM) (Table 1). It is remarkable that both the streamflow to the reservoir and the AF showed common oscillations with ENSO at fi-equencies between 0.016 and 0.035 cycles month These frequencies are very close to the two main periods of ENSO (the quasi-biennial and quasi-quadrennial periods) [56]. Although we do not have a mechanistic explanation for this teleconnection (in fact the extratropical influence of ENSO is a hot topic in climate research, Merkel and Latif [57]), it is certainly difficult to propose an alternative explanation for the oscillations in AF and stream-flow observed at these frequencies. 

Figure 4.19 Electrospray spectra of a protein (a) after transformation, and (b) after maximum entropy processing. From applications literature published by Micromass UK Ltd, Manchester, UK, and reproduced with permission.

Buck, B. and Macaulay, V. A. (Eds), Maximum Entropy in Action, Oxford University Press, Oxford, UK, 1991,... 

The raw electrospray spectrum obtained is shown in Figure 5.14. Maximum entropy processing of these data yielded the spectrum shown in Figure 5.15, which shows the presence of two species with molecular masses of 14293.6 and 14 309.6 Da, with the latter being attributed to partial oxidation of the parent protein. 

Maximum entropy A computer algorithm used to predict the theoretical signal from which that observed in a spectrum has been derived. Used in conjunction with electrospray ionization to enhance the quality of the spectra obtained. 

C14-0123. The following diagram represents two flasks connected by a valve. Each flask contains a different gas. (a) If the valve is opened, the two gases can move back and forth between the two flasks. Redraw the figure to show the system at maximum entropy, (b) Redraw the figure in a way that shows the state of the system at its lowest possible entropy. (The substances need not remain gaseous.)... 

As we all know from thermodynamics, closed systems in equilibrium have minimum free energy and maximum entropy. If such a system were brought out of equilibrium, i.e. to a state with lower entropy and higher free energy, it would automatically decay to the state of equilibrium, and it would lose all information about its previous states. A system s tendency to return to equilibrium is given by its free energy. An example is a batch reaction that is run to completion. 

Another class of methods such as Maximum Entropy, Maximum Likelihood and Least Squares Estimation, do not attempt to undo damage which is already in the data. The data themselves remain untouched. Instead, information in the data is reconstructed by repeatedly taking revised trial data fx) (e.g. a spectrum or chromatogram), which are damaged as they would have been measured by the original instrument. This requires that the damaging process which causes the broadening of the measured peaks is known. Thus an estimate g(x) is calculated from a trial spectrum fx) which is convoluted with a supposedly known point-spread function h(x). The residuals e(x) = g(x) - g(x) are inspected and compared with the noise n(x). Criteria to evaluate these residuals are Maximum Entropy (see Section 40.7.2) and Maximum Likelihood (Section 40.7.1). 

---