Markov master equation

In the special case where the site energies are random fluctuations, this is the Anderson model [20,21]. It is well known that Anderson used this model to prove that randomness makes a crystal become an insulating material. Anderson localization is subtly related to subdiffusion, and consequently this important phenomenon can be interpreted as a form of anomalous diffusion, in conflict with the Markov master equation that is frequently adopted as the generator of ordinary diffusion. It is therefore surprising that this is essentially the same Hamiltonian as that adopted by Zwanzig for his celebrated derivation of the van Hove and, hence, of the Pauli master equation. 

Thus, Eq. (55) becomes identical to Eq. (2), thereby recovering an important result obtained many years ago [38] A Poisson process, with /(f) given by an exponential function of t, yields a Markov master equation. 

Thus, we recover the Markov master equation (122), and with it [50], taking the limit 7W < I, we obtain the important equation... 

Although Eq. (16) has a very simple form, the quantities a(x, y, t) may be very complicated. This is why the probabilistic master equation, Eq. (16), appears much simpler than the physical master equation which will be discussed presently. The equations which describe the time development of the conditional probabilities w2 are also obtained straightforwardly from Eqs. (lb) and (2). The results for a Markov process are... 

All that remains to be done for determining the fluctuation spectrum is to compute the conditional average, Eq. (31). However, this involves the full equations of motion of the many-body system and one can at best hope for a suitable approximate method. There are two such methods available. The first method is the Master Equation approach described above. Relying on the fact that the operator Q represents a macroscopic observable quantity, one assumes that on a coarse-grained level it constitutes a Markov process. The microscopic equations are then only required for computing the transition probabilities per unit time, W(q q ), for example by means of Dirac s time-dependent perturbation theory. Subsequently, one has to solve the Master Equation, as described in Section TV, to find both the spectral density of equilibrium fluctuations and the macroscopic phenomenological equation. 

On the other hand, (1.4) may be interpreted as to mean that any solution with initial condition P(y, t0) = 5(y — y0) is identical with the transition probability Pi i y> 11y0, to)- In fact, the master equation derived in the next chapter for Markov processes is of this type. Yet it cannot guarantee Markovian character, inasmuch as it still does not say anything about the higher distribution functions. ... 

The operator Q in the master equation is linear. It can be seen that the transition probability of a Markov process cannot obey a nonlinear equation of the form (1.4). The argument is similar to the one used by D. Polder, Philos. Mag. 45, 69 (1954). 

The master equation is an equivalent form of the Chapman-Kolmogorov equation for Markov processes, but it is easier to handle and more directly related to physical concepts. It will be the pivot of most of the work in this book. 

Consider a Markov process, which for convenience we take to be homogeneous, so that we may write Tx for the transition probability. The Chapman-Kolmogorov equation (IV.3.2) for Tx is a functional relation, which is not easy to handle in actual applications. The master equation is a more convenient version of the same equation it is a differential equation obtained by going to the limit of vanishing time difference t. For this purpose it is necessary first to ascertain how Tx> behaves as x tends to zero. In the previous section it was found that TX (y2 yl) for small x has the form ... 

This interpretation of the master equation means that is has an entirely different role than the Chapman-Kolmogorov equation. The latter is a nonlinear equation, which results from the Markov character, but contains no specific information about any particular Markov process. In the master equation, however, one considers the transformation probabilities as given by the specific system, and then has a linear equation for the probabilities which determine the (mesoscopic) state of that system. 

Exercise. If W is a 2 x 2 matrix the exponential in (2.4) can be evaluated directly by expanding in powers of W. Use this method for solving the master equation of the dichotomic Markov process. 

Exercise. For Markov processes that are not stationary or homogeneous one also has a forward, or master equation and a backward equation,... 

Once a connection is made it is discontinued at a random moment, rn = an. (This somewhat unrealistic assumption is needed for the Markov character.) The master equation is... 

Solving this equation means determining the probability to find at t > 0 the system in (i, r) when it was at t = 0 in (i0, r0). This problem decomposes into two successive steps first find how the molecule jumps among the levels regardless of r, and subsequently add on the behavior in r. This is the reason why we use the name composite Markov process for any random process obeying a master equation of the type (7.4). ... 

The Fokker-Planck equation is a special type of master equation, which is often used as an approximation to the actual equation or as a model for more general Markov processes. Its elegant mathematical properties should not obscure the fact that its application in physical situations requires a physical justification, which is not always obvious, in particular not in nonlinear systems. 

Markov processes whose master equation has the form (1.1) have been called continuous , because it can be proved that their sample functions are continuous (with probability 1). This name has sometimes led to the erroneous idea that all processes with a continuous range are of this type and must therefore obey (1.1). 

Another, even simpler choice for is the two-valued Markov process (IV.2.3). The joint master equation reduces to two coupled equations for the pair of singlevariable functions (y, 1, t) = P (y, t),... 

In terms of the master equation for the Markov process the formal kinetics is nothing but the mean-field theory where the fluctuation terms like that on the r.h.s. of equation (2.2.43) are neglected. Strictly speaking, the macroscopic description, equation (2.1.2), were correct if the fluctuation terms vanished as V —> oo. In a general case the function P(N, t) does not satisfy the Poisson distribution [16, 27] in particular, °N (N> ... 

Note here that the relation between mesoscopic and microscopic approaches is not trivial. In fact, the former is closer to the macroscopic treatment (Section 2.1.1) which neglects the structural characteristics of a system. Passing from the micro- to meso- and, finally, to macroscopic level we loose also the initial statement of a stochastic model of the Markov process. Indeed, the disadvantages of deterministic equations used for rather simplified treatment of bimolecular kinetics (Section 2.1) lead to the macro- and mesoscopic models (Section 2.2) where the stochasticity is kept either by adding the stochastic external forces (Section 2.2.1) or by postulating the master equation itself for the relevant Markov process (Section 2.2.2). In the former case the fluctuation source is assumed to be external, whereas in the latter kinetics of bimolecular reaction and fluctuations are coupled and mutually related. Section 2.3.1.2 is aimed to consider the relation between these three levels as well as to discuss problem of how determinicity and stochasticity can coexist. 

A structure of the obtained set of equations derived by us in [81, 86] is very close to the famous BBGKI set of equations widely used in the statistical physics of dense gases and liquids [76]. Therefore, we presented the master equation of the Markov process in a form of the infinite set of deterministic coupled equations for averages (equation (2.3.34)). Practical use of these equations requires us to reduce them, retaining the joint correlation functions only. 

Note that willing to stress the relation between many-particle densities and master equation for the Markov process, we followed the formalism presented by us [81] rather than that used in the pioneering papers by Waite [84, 94, 95] and Leibfried [96], as well as in more recent studies [82, 97, 103-105] where... 

Given all these quantities, the master equation could be written for a set of reactions (2.1.27) as the Markov process. (Actual choice of the functions [Pg.474]

To formulate this stochastic model in terms of concentrations and joint correlation functions only, i.e., in a manner we used earlier in Chapters 2, 4 and 5, it is convenient to write down a master equation of the Markov process under study in a form of the infinite set of coupled equations for many-point densities. Let us write down the first equations for indices (m + m ) = 1 ... 

For a continuous Markov process, the master equation is of the form... 

For many physical applications, modeled by a homogeneous Markov process in time and space, the rate of transition is time independent and depends only on the difference of the starting and arriving states. Therefore, one can see that the master equation is given by... 

Although a theoretical approach has been desecrated as to how one can apply the generalized coupled master equations to deal with ultrafast radiationless transitions taking place in molecular systems, there are several problems and limitations to the approach. For example, the number of the vibrational modes is limited to less than six for numerical calculations. This is simply just because of the limitation of the computational resources. If the efficient parallelization can be realized to the generalized coupled master equations, the limitation of the number of the modes can be relaxed. In the present approach, the Markov approximation to the interaction between the molecule and the heat bath mode has been employed. If the time scale of the ultrashort measurements becomes close to the characteristic time of the correlation time of the heat bath mode, the Markov approximation cannot be applicable. In this case, the so-called non-Markov treatment should be used. This, in turn, leads to a more computationally demanding task. Thus, it is desirable to develop a new theoretical approach that allows a more efficient algorithm for the computation of the non-Markov kernels. Another problem is related to the modeling of the interaction between the molecule and the heat bath mode. In our model, the heat bath mode is treated as... 

Stochastic analysis presents an alternative avenue for dealing with the inherently probabilistic and discontinuous microscopic events that underlie macroscopic phenomena. Many processes of chemical and physical interest can be described as random Markov processes.1,2 Unfortunately, solution of a stochastic master equation can present an extremely difficult mathematical challenge for systems of even modest complexity. In response to this difficulty, Gillespie3-5 developed an approach employing numerical Monte Carlo... 

Of course the diagonal density elements pj 1 (f), p22(t) are identified with the probabilities p (t) and pi(t), respectively. One of the major tasks of this review is to point out the difficulties that are currently met with the extension of this interpretation to the case where the Markov condition of the ordinary Pauli master equation does not apply. 

---