Markovian jump process

Discrete state space stochastic models of chemical reactions can be identified with the Markovian jump process. In this case the temporal evolution can be described by the master equation ... 

Fig. 5.11 Interrelations between different types of processes (MPP, Markov population process CCR, complex chemical reaction SBD, simple birth and death process S, V, simple Markovian jump processes and models of reactions in the scalar and vector case respectively).

The connection between the stochastic cell model and the usual deterministic model of reaction-diffusion systems were given by Kurtz for the homogeneous case in the same spirit. To give the relationship among a Markovian jump process and the solution p(r, i) of the deterministic model a law of large numbers and a central limit theorem hold (Arnold Theodoso-pulu, 1980 Arnold 1980). 

Both deterministic and stochastic models can be defined to describe the kinetics of chemical reactions macroscopically. (Microscopic models are out of the scope of this book.) The usual deterministic model is a subclass of systems of polynomial differential equations. Qualitative dynamic behaviour of the model can be analysed knowing the structure of the reaction network. Exotic phenomena such as oscillatory, multistationary and chaotic behaviour in chemical systems have been studied very extensively in the last fifteen years. These studies certainly have modified the attitude of chemists, and exotic begins to become common . Stochastic models describe both internal and external fluctuations. In general, they are a subclass of Markovian jump processes. Two main areas are particularly emphasised, which prove the importance of stochastic aspects. First, kinetic information may be extracted from noise measurements based upon the fluctuation-dissipation theorem of chemical kinetics second, noise may change the qualitative behaviour of systems, particularly in the vicinity of instability points. 

The CDS model, which is a time-homogenous Markovian jump process, proved to be relevant for describing composition fluctuation phenomena both around and out of equilibrium. While the CDS model is not well-founded from microscopic point of view, the availability of new techniques for the study of fast reactions (e.g. Jonah, this volume) even at the femtosecond scale makes necessary to set up coupled microscopic - mesoscopic models. Earlier works (e.g.Gaveau and Moreau 1985, Borgis et al 1986, Moreau and Gaveau 1987) emphasizing the existence of non-Markovian collision processes tended into this direction. 

Markovian jump processes are characterized by exponential waiting time distribution (Doob 1953,pp. 244). 

Mixing models based on the CD model have discrete jumps in the composition vector, and thus cannot be represented by a diffusion process (i.e., in terms of and B ). Instead, they require a generalization of the theory of Markovian random processes that encompasses jump processes136 (Gardiner 1990). The corresponding governing equation... 

One can derive Eqn. (12,12) in a more fundamental way by starting the statistical approach with the (Markovian) master equation, assuming that the jump probabilities obey Boltzmann statistics on the activation saddle points. Salje [E. Salje (1988)] has discussed the following general form of a kinetic equation for solid state processes... 

In order to simplify the description of this system one neglects the fast dynamics in the potential wells and considers only the transitions from one well to the other which happen on a much slower time scale. Under the assumption that the potential barrier AU between the two wells is large compared to the noise strength D and the relaxation in the wells is fast compared to the time scale of the jumps between the wells, the transitions can be considered as a rate process. Such a rate process has a probability per unit time to cross the barrier, which is independent on the time which has elapsed since the last crossing event. The resulting dynamics in the reduced discrete phase space which consists just of two discrete states left and right is thus still a Markovian one, i.e. the present state determines the future evolution to a maximal extent. 

In a previous section reference was made to the random walk problem (Montroll and Schlesinger [1984], Weiss and Rubin [1983]) and its application to diffusion in solids. Implicit in these methods are the assnmptions that particles hop with a fixed jump distance (for example between neighboring sites on a lattice) and, less obviously, that jumps take place at fixed equal intervals of time (discrete time random walks). In addition, the processes are Markovian, that is the particles are without memory the probability of a given jump is independent of the previous history of the particle. These assumptions force normal or Gaussian diffusion. Thus, the diffusion coefficient and conductivity are independent of time. 

Markov states. Consequently the original state variables and the states of the auxiliary pure-jump stochastic process are jointly Markovian. The jumps have to be defined in such a way that the actual impulse (i.e., the jump in the velocity response Z2(0) only occurs if there is a jump between some particular Markov states. 

---