Markov dynamics

A very important application of the Markov dynamics is random walk. In the special case of random walk/(x) = 0 and g(x) = 1, then the diffusion equation for a random walk in one dimension is... 

Beside ergodicity, another demand for correct statistical sampling is to ensure that the probability distributionp (X) associated with the desired statistical ensemble is independent of time. This can only be achieved in the simulation, if the relevant part of the phase space is sampled sufficiently efficiently to allow for quick convergence toward a stable or, more precisely, stationary estimate for/>(X). In most of the Monte Carlo methods, the simulation follows a Markov dynamics, i.e., the update of a given conformation X to a new one X is not influenced by the history that led to X, i.e., the dynamics does not possess an explicit memory. Such a Markov process can be described by the master equation ... 

The key quantity in barrier crossing processes in tiiis respect is the barrier curvature Mg which sets the time window for possible influences of the dynamic solvent response. A sharp barrier entails short barrier passage times during which the memory of the solvent environment may be partially maintained. This non-Markov situation may be expressed by a generalized Langevin equation including a time-dependent friction kernel y(t) [ ]... 

While static Monte Carlo methods generate a sequence of statistically independent configurations, dynamic MC methods are always based on some stochastic Markov process, where subsequent configurations X of the system are generated from the previous configuration X —X —X" — > with some transition probability IF(X —> X ). Since to a large extent the choice of the basic move X —X is arbitrary, various methods differ in the choice of the basic unit of motion . Also, the choice of transition probability IF(X — > X ) is not unique the only requirement is that the principle... 

Here Jta(x) denotes the a-th component of the stationary vector x of the Markov chain with transition matrix Q whose elements depend on the monomer mixture composition in microreactor x according to formula (8). To have the set of Eq. (24) closed it is necessary to determine the dependence of x on X in the thermodynamic equilibrium, i.e. to solve the problem of equilibrium partitioning of monomers between microreactors and their environment. This thermodynamic problem has been solved within the framework of the mean-field Flory approximation [48] for copolymerization of any number of monomers and solvents. The dependencies xa=Fa(X)(a=l,...,m) found there in combination with Eqs. (24) constitute a closed set of dynamic equations whose solution permits the determination of the evolution of the composition of macroradical X(Z) with the growth of its length Z, as well as the corresponding change in the monomer mixture composition in the microreactor. 

The state of the entire system at time t is described by the /V-particle phase space probability density function, P(x/V, t). In MPC dynamics the time evolution of this function is given by the Markov chain,... 

In addition to the fact that MPC dynamics is both simple and efficient to simulate, one of its main advantages is that the transport properties that characterize the behavior of the macroscopic laws may be computed. Furthermore, the macroscopic evolution equations can be derived from the full phase space Markov chain formulation. Such derivations have been carried out to obtain the full set of hydrodynamic equations for a one-component fluid [15, 18] and the reaction-diffusion equation for a reacting mixture [17]. In order to simplify the presentation and yet illustrate the methods that are used to carry out such derivations, we restrict our considerations to the simpler case of the derivation of the diffusion equation for a test particle in the fluid. The methods used to derive this equation and obtain the autocorrelation function expression for the diffusion coefficient are easily generalized to the full set of hydrodynamic equations. 

The hydrodynamic equations can be derived from the MPC Markov chain dynamics using projection operator methods analogous to those used to obtain... 

Markov chain formulation, multiparticle collision dynamics ... 

Markov decay time, Monte Carlo heat flow simulation, nonequilibrium molecular dynamics, 80-81... 

In the most general case the diffusive Markov process (which in physical interpretation corresponds to Brownian motion in a field of force) is described by simple dynamic equation with noise source ... 

The Markov processes associated with quantum star graphs correspond to systems of weakly coupled edges. Its dynamical properties are determined by the spectrum of the stochastic matrix associated with (14) which is highly degenerate and can be given explicitly (Kottos and Smilansky 1999), that is,... 

The authors then ask the following question Do there exist deterministic dynamical systems that are, in a precise sense, equivalent to a monotonous Markov process The question can be reformulated in a more operational way as follows Does there exist a similarity transformation A which, when applied to a distribution function p, solution of the Liouville equation, transforms the latter into a function p that can also be interpreted as a distribution function (probability density) and whose evolution is governed by a monotonous Markov process An affirmative answer to this question requires the following conditions on A (MFC) ... 

The dynamical randomness of this Markov chain is characterized by the Kolmogorov-Sinai entropy per unit time ... 

Figure 18. The dynamical entropies (126) and (127) as well as the entropy production (128) for the three-state Markov chain defined by the matrix (125) of transition probabilities versus the parameter a. The equilibrium corresponds to the value a — Ij i. The process is perfectly cyclic at a = 0 where the path is. .. 123123123123. .. and the Kohnogorov-Sinai entropy h vanishes as a...

Analysis of the poly(methyl methacrylate) sequences obtained by anionic polymerization was undertaken at the tetrad level in terms of two different schemes (10) one, a second-order Markov distribution (with four independent conditional probabilities, Pmmr Pmrr, Pmr Prrr) (44), the other, a two-state mechanism proposed by Coleman and Fox (122). In this latter scheme one supposes that the chain end may exist in two (or more) different states, depending on the different solvation of the ion pair, each state exerting a specific stereochemical control. A dynamic equilibrium exists between the different states so that the growing chain shows the effects of one or the other mechanism in successive segments. The deviation of the experimental data from the distribution calculated using either model is, however, very small, below experimental error, and, therefore, it is not possible to make a choice between the two models on the basis of statistical criteria only. 

The preceding discussion applied implicitly to what we classify as dynamical simulations — namely, those simulations in which all correlations in the final trajectory arise because each configuration is somehow generated from the previous one. This time-correlated picture applies to a broad class of algorithms MD, Langevin and Brownian dynamics, as well as traditional Monte Carlo (MC, also known as Markov-chain Monte Carlo). Even though MC may not lead to true physical dynamics, all the correlations are sequential. 

Chodera, J.D., Singhal, N., Pande, V.S., Dill, K.A., Swope, W.C. Automatic discovery of metastable states for the construction of Markov models of macromolecular conformational dynamics. J. Chem. Phys. 2007, 126, 155101-17. 

We can understand better this asymptotics by using the Markov chain language. For nonseparated constants a particle in has nonzero probability to reach and nonzero probability to reach A, . The zero-one law in this simplest case means that the dynamics of the particle becomes deterministic with probability one it chooses to go to one of vertices A, A3 and to avoid another. Instead of branching, A2 A and A2 A3, we select only one way either A2 A] or A2 A3. Graphs without branching represent discrete dynamical systems. 

Therefore, the simplest procedure to get the stochastic description of the reaction leads to the rather complicated set of equations containing phenomenological parameters / (equation (2.2.17)) with non-transparent physical meaning. Fluctuations are still considered as a result of the external perturbation. An advantage of this approach is a useful analogy of reaction kinetics and the physics of equilibrium critical phenomena. As is well known, because of their nonlinearity, equations (2.1.40) reveal non-equilibrium bifurcations [78, 113]. A description of diffusion-controlled reactions in terms of continuous Markov process - equation (2.2.15) - makes our problem very similar to the static and dynamic theory of critical phenomena [63, 87]. When approaching the bifurcation points, the systems with reactions become very sensitive to the environment fluctuations, which can even produce new nonequilibrium transitions [18, 67, 68, 90, 108]. The language developed in the physics of critical phenomena can be directly applied to the processes in spatially extended systems. 

The single relaxation time approximation corresponds to a stochastic model in which the fluctuating force on a molecule has a Lorentzian spectrum. Thus if the fluctuating force is a Gaussian-Markov process, it follows that the memory function must have this simple form.64 Of course it would be naive to assume that this exponential memory will accurately account for the dynamical behavior on liquids. It should be regarded as a simple model which has certain qualitative features that we expect real memory functions to have. It decays to zero and, moreover, is of a sufficiently simple mathematical form that the velocity autocorrelation function,... 

The knowledge of the two-minima energy surface is sufficient theoretically to determine the microscopic and static rate of reaction of a charge transfer in relation to a geometric variation of the molecule. In practice, the experimental study of the charge-transfer reactions in solution leads to a macroscopic reaction rate that characterizes the dynamics of the intramolecular motion of the solute molecule within the environment of the solvent molecules. Stochastic chemical reaction models restricted to the one-dimensional case are commonly used to establish the dynamical description. Therefore, it is of importance to recall (1) the fundamental properties of the stochastic processes under the Markov assumption that found the analysis of the unimolecular reaction dynamics and the Langevin-Fokker-Planck method, (2) the conditions of validity of the well-known Kramers results and their extension to the non-Markovian effects, and (3) the situation of a reaction in the absence of a potential barrier. 

K.K. Yin, H. Yang, P. Daoutidis, G.G. Yin, Simulation of population dynamics using continuous-time finite state Markov chains, Compt. Chem. Eng. 27 (2003) 235-249. 

To account for the radiative decay of CC excited states we consider the density operator p, Eq. (35), reduced to the CC solvent states. It is a standard task of dissipative quantum dynamics to derive an equation of motion for p with a second order account for the CC-photon coupling, Eq. (24) (see, for example, [40]). Focusing on the excited CC-state contribution, in the most simple case (Markov and secular approximation) we expect the following equation of motion... 

---