Distribution of maximal entropy

One can show that the result, Eq. (2.15), that (2.12) is the distribution of maximal entropy implies also that P(x) is more uniform than Q(x). 

Experimental and computational results often do deviate from the distributions of maximal entropy subject only to a given total strength. Consider the following simple modification. The spectrum can be regarded as the set of expectation values of the state Pi... 

We now impose not only a given overall strength but also a given envelope. The average local intensity is thus also specified. This requires the introduction of the set of N constraints (3.13), each of which is assigned the Lagrange multiplier yf. The distribution of maximal entropy subject to the three constraints (1)—(3)is... 

The amplitudes of the different lines in the spectrum can be regarded as the components of the optically prepared bright state in the basis of the eigenstates of the system, cf. Eqs. (2)-(3). Already in Sec. II we had several occasions to note that the bright state behaves not unlike a random vector. One can therefore ask what the spectrum will be if we make the approximation that these components are truly random. This requires us to specify, in a technical sense, what we mean by random. This is where entropy comes in. By random we will mean that the distribution of the amplitudes be as uniform as possible and, as such, be a distribution of maximal entropy. 

The details of determining a distribution of maximal entropy are spelled out in a number of reviews. Here we just give an example suppose the dynamic constraint is the final vibrational energy. This means that the dynamics imposes on the distribution of final states a mean value of the final vibrational energy... 

In some circles the approach that we follow, that of looking for a distribution of maximal entropy. Section 6.4.2.1, is questioned. This occurs because people sometimes overlook the need to include a prior distribution in Eq. (6.32). In this case it follows that in the absence of constraints all groups of outcomes are equalty probable. Very often this is manifestly not reasonable and correctly so because different groups can differ in how many mutually exclusive outcomes are members of the group. 

The prior distribution is often not what is observed, and there can be extreme deviations from it. [1, 3, 23] By our terminology this means that the dynamics do impose constraints on what can happen. How can one explicitly impose such constraints without a full dynamical computation At this point I appeal again to the reproducibility of the results of interest as ensured by the Monte Carlo theorem. The very reproducibility implies that much of the computed detail is not relevant to the results of interest. What one therefore seeks is the crudest possible division into cells in phase space that is consistent with the given values of the constraints. This distribution is known as one of maximal entropy". 

There are two ways to implement this program. One is to directly discuss the distribution of final states. This is known as surprisal analysis . In this simpler procedure one does not ask how this distribution came about Instead, one seeks the coarsest or most statistical" (= of maximal entropy) distribution of final states, subject to constraints. The last proviso is, of course, essential. If no constraints axe imposed, one will obtain as an answer the prior distribution. It is the constraints that generate a distribution which contains just that minimal dynamical detail as is necessary to generate the answers of interest. Few simple and physically obvious constraints are often sufficient [1, 3, 23] to account for even extreme deviations from the prior distribution. 

The reason we employ two rather distinct methods of inquiry is that neither, by itself, is free of open methodological issues. The method of molecular dynamics has been extensively applied, inter alia, to cluster impact. However, there are two problems. One is that the results are only as reliable as the potential energy function that is used as input. For a problem containing many open shell reactive atoms, one does not have well tested semiempirical approximations for the potential. We used the many body potential which we used for the reactive system in our earlier studies on rare gas clusters containing several N2/O2 molecules (see Sec. 3.4). The other limitation of the MD simulation is that it fails to incorporate the possibility of electronic excitation. This will be discussed fmther below. The second method that we used is, in many ways, complementary to MD. It does not require the potential as an input and it can readily allow for electronically excited as well as for charged products. It seeks to compute that distribution of products which is of maximal entropy subject to the constraints on the system (conservation of chemical elements, charge and... 

To implement the procedure of maximal entropy, we need to write down the entropy. Here care need be exercised since the usual expressions for the entropy are only valid for that distribution which, in the absence of all but the normalization constraint, is uniform. We suggest that this natural distribution is not necessarily P(y). Consider first the case where all bound states and the transition operator are real. Then y = x2 where x is real. In the absence of any constraints we require that the natural distribution remain unchanged under an orthogonal change of the basis set. The distribution that is unchanged by a rotation of x is the distribution uniform in x. The alternative, a distribution uniform in y, will not stay uniform after a rotation of the basis. 

Consider first the limit where no dynamical input is given. Then only the value of C(t) at / = 0 is imposed as a constraint in determining P(yf) by the procedure of maximal entropy. In this case all the Lagrange multipliers have exactly the same value, namely (x = 1/X.o and so all the distributions P(yf) are the same. One can therefore explicitly carry out the sum over final states in (102), with the result... 

The commercially available software (Maximum Entropy Data Consultant Ltd, Cambridge, UK) allows reconstruction of the distribution a.(z) (or f(z)) which has the maximal entropy S subject to the constraint of the chi-squared value. The quantified version of this software has a full Bayesian approach and includes a precise statement of the accuracy of quantities of interest, i.e. position, surface and broadness of peaks in the distribution. The distributions are recovered by using an automatic stopping criterion for successive iterates, which is based on a Gaussian approximation of the likelihood. 

The distribution of fragment size is computed as one of the maximal entropy subject to two constraints (1) conservation of matter ... 

Collins, D. M. Extrapolative filtering. I. Maximization of resolution for onedimensional positive density functions. Acta Cryst. A34, 533-541 (1978). Bricogne, G. A Bayesian statistical theory of the phase problem. I. A multichannel maximum-entropy formalism for constructing generalized joint probability distributions of structure factors. Acta Cryst. A44, 517-545 (1988). 

Internally equilibrated subsystems, which act as free energy reservoirs, are already as random as possible given their boundary conditions, even if they are not in equilibrium with one another because of some bottleneck. Tlius, the only kinds of perturbation that can arise and be stabilized when they are coupled are those that make the joint system less constrained than the subsystems originally were. (This is Boltzmann s H-theorem [9] only a less constrained joint system has a liigher maximal entropy than die sum of entropies from the subsystems independently and can stably adopt a different form.) The flows that relax reservoir constraints are thermochemical relaxation processes toward the equilibrium state for tlte joint ensemble. The processes by wliich such equilibration takes place are by assumption not reachable within the equilibrium distribution of either subsystem. As the nature of the relaxation phenomenon often depends on aspects of the crosssystem coupling that are much more specific than the constraints that define either reservoir, they are often correspondingly more complex than the typical processes... 

The index i is a superindex that refers to a particular realization of both E and N. If we now follow the same procedure used earlier to derive the canonical distribution by maximizing the entropy, it is found that the resulting probabilities are of the form... 

For a dimension-bearing variable, one generally expects SS oc Sy/y or S(y) oc lny. Hence we can impose as a constraint on the maximization of the entropy. This will lead to a y-square distribution where now (v - l)/2 is the Lagrange multiplier for this additional constraint. The value of v is now to be determined as usual, by equating the value of as determined from (2.25)... 

The case of a truly random bright state is likely to be a limiting one. In reality, the dynamics do constrain the time evolution of the bright state. What one therefore needs is a prescription for dealing with situations which are not fully random. Entropy continues to provide a convenient tool because rather than look for that distribution which is as random as possible one can instead specify that distribution which is as uniform as it can be subject to the input from the dynamics. In technical terms, we seek that distribution of amplitudes whose entropy is maximal, but the maximum is subject to auxiliary conditions. These additional conditions (or constraints) are to be provided by the dynamics. The result is a spectrum that is consistent with the given dynamical input and is otherwise the result of a maximally uniform set of amplitudes. Section IV.A.2 provides a more technical version of these considerations. 

---