Markov processes diffusion process

The aim of this chapter is to describe approaches of obtaining exact time characteristics of diffusion stochastic processes (Markov processes) that are in fact a generalization of FPT approach and are based on the definition of characteristic timescale of evolution of an observable as integral relaxation time [5,6,30—41]. These approaches allow us to express the required timescales and to obtain almost exactly the evolution of probability and averages of stochastic processes in really wide range of parameters. We will not present the comparison of these methods because all of them lead to the same result due to the utilization of the same basic definition of the characteristic timescales, but we will describe these approaches in detail and outline their advantages in comparison with the FPT approach. 

A continuous Markov process (also known as a diffusive process) is characterized by the fact that during any small period of time At some small (of the order of %/At) variation of state takes place. The process x(t) is called a Markov process if for any ordered n moments of time t < < t < conditional probability density depends only on the last fixed value ... 

In the most general case the diffusive Markov process (which in physical interpretation corresponds to Brownian motion in a field of force) is described by simple dynamic equation with noise source ... 

The first approach to obtain exact time characteristics of Markov processes with nonlinear drift coefficients was proposed in 1933 by Pontryagin, Andronov, and Vitt [19]. This approach allows one to obtain exact values of moments of the first passage time for arbitrary time constant potentials and arbitrary noise intensity moreover, the diffusion coefficient may be nonlinear function of coordinate. The only disadvantage of this method is that it requires an artificial introducing of absorbing boundaries, which change the process of diffusion in real smooth potentials. 

Brownian motion of a constrained system of N point particles may also be described by an equivalent Markov process of the Cartesian bead positions R (f),..., R (f). The constrained diffusion of the Cartesian coordinates may be characterized by a Cartesian drift velocity vector and diffusivity tensor... 

The connection between a diffusion equation and a corresponding Markov diffusion process may be established through expressions for drift velocities and diffusitivies. The drift velocity for both unconstrained and constrained systems may be expressed in an arbitrary system of coordinates in the generic form... 

In this section, we consider the description of Brownian motion by Markov diffusion processes that are the solutions of corresponding stochastic differential equations (SDEs). This section contains self-contained discussions of each of several possible interpretations of a system of nonlinear SDEs, and the relationships between different interpretations. Because most of the subtleties of this subject are generic to models with coordinate-dependent diffusivities, with or without constraints, this analysis may be more broadly useful as a review of the use of nonlinear SDEs to describe Brownian motion. Because each of the various possible interpretations of an SDE may be defined as the limit of a discrete jump process, this subject also provides a useful starting point for the discussion of numerical simulation algorithms, which are considered in the following section. 

Throughout this section, we will use the notation X (t),..., X t) to denote a unspecified set of L Markov diffusion processes when discussing mathematical properties that are unrelated to the physics of constrained Brownian motion, or that are not specific to a particular set of variables. The variables refer specifically to soft coordinates, generalized coordinates for a system of N point particles, and Cartesian particle positions, respectively. The generic variables X, ..., X will be indexed by integer variables a, p,... = 1,...,L. 

In both the Ito and Stratonovich formulations, the randomness in a set of SDEs is generated by an auxiliary set of statistically independent Wiener processes [12,16]. The solution of an SDE is defined by a hmiting process (which is different in different interpretations) that yields a unique solution to any stochastic initial value problem for each possible reahzation of this underlying set of Wiener processes. A Wiener process W t) is a Gaussian Markov diffusion process for which the change in value W t) — W(t ) between any two times t and t has a mean and variance... 

This algorithm differs from the Markov process used in Section IX to define a kinetic SDE only in that an explicitly predicted midstep position, rather than an implicitly defined midstep position, is used to calculate the midstep velocity for the final update. To the accuracy needed to calculate the drift velocity and diffusivity, the analyses of the explicit and implicit midstep schemes are identical. As a result, the preceding calculation of the diffusivity and drift for a kinetic SDE also applies to this algorithm. 

So far we studied the first passage of Markov processes such as described by the Smoluchowski equation (1.9). On a finer time scale, diffusion is described by the Kramers equation (VIII.7.4) for the joint probability of the position X and the velocity V. One may still ask for the time at which X reaches for the first time a given value R, but X by itself is not Markovian. That causes two complications, which make it necessary to specify the first-passage problem in more detail than for diffusion. 

Therefore, the simplest procedure to get the stochastic description of the reaction leads to the rather complicated set of equations containing phenomenological parameters / (equation (2.2.17)) with non-transparent physical meaning. Fluctuations are still considered as a result of the external perturbation. An advantage of this approach is a useful analogy of reaction kinetics and the physics of equilibrium critical phenomena. As is well known, because of their nonlinearity, equations (2.1.40) reveal non-equilibrium bifurcations [78, 113]. A description of diffusion-controlled reactions in terms of continuous Markov process - equation (2.2.15) - makes our problem very similar to the static and dynamic theory of critical phenomena [63, 87]. When approaching the bifurcation points, the systems with reactions become very sensitive to the environment fluctuations, which can even produce new nonequilibrium transitions [18, 67, 68, 90, 108]. The language developed in the physics of critical phenomena can be directly applied to the processes in spatially extended systems. 

Levy diffusion is a Markov process corresponding to the conditions established by the ordinary random walk approach with the random walker making jumps at regular time values. To explain why the GME, with the assumption of Eq. (112), yields Levy diffusion, we notice [50] that the waiting time distribution is converted into a transition probability n(x) through... 

Model (4.79) describes an evolutionary process, which results from the coupling of a Markov chain assistance with some individual diffusion processes. This model is well known in the study of the coupling of a chemical reaction with diffusion phenomena [4.5, 4.6, 4.34, 4.35]. The models described by relations (4.63) and (4.79) can still be particularized or generalized. As an example, we can notice that other types of models can be suggested if we consider that the values of Ukj are functions of z or t or Pk(z,r) in Eq.(4.79). However, it is important to observe that the properties of the Markov type connections cannot be considered when kj =f(PK(z. c))-... 

If F(X,v) = v or if v is the device speed, then the stochastic differential equation (4.80) shows that the state of the device is a function which depends on position and speed. The device passes from one speed to another with the rules defined by a diffusion process and with an average value mj(v) and a variance probability densities of the coupled Markov process (X(t),v(t)) - written p= p(t,X,v,Xo,Vo) - should verify the following equation ... 

To analyse bond breakage under steady loading, we take advantage of the enormous gap in time scale between the ultrafast Brownian diffusion (r 10 — 10 s) and the time frame of laboratory experiments ( 10 s to min). This means that the slowly increasing force in laboratory experiments is essentially stationary on the scale of the ultrafast kinetics. Thus, dissociation rate merely becomes a function of the instantaneous force and the distribution of rupture times can be described in the limit of large statistics by a first-order (Markov) process with time-dependent rate constants. As force rises above the thermal force scale, i.e. rj-t> k T/x, the forward transition... 

For translational long-range jump diffusion of a lattice gas the stochastic theory (random walk, Markov process and master equation) [30] eventually yields the result that Gg(r,t) can be identified with the solution (for a point-like source) of the macroscopic diffusion equation, which is identical to Pick s second law of diffusion but with the tracer (self diffusion) coefficient D instead of the chemical or Fick s diffusion coefficient. 

The same result is obtained for the mean quadratic displacement of a freely diffusing particle and eludes to the same underlying physical principle, namely, the statistics of Markov processes. 

In the mathematical literature, X (t ) is called a semi-Markov process associated with the two-component Markov chain X , T ), a Markov renewal process [218]. As discussed in Sect. 2.3, the CTRW model is a standard approach for studying anomalous diffusion [298]. 

Mattingly, J., Stuart, A. Geometric ergodicity of some hypo-elliptic diffusions for particle motions. Markov Process. Relat. Fields 8, 199-214 (2002)... 

Rogers, L., Wdliams, D. Diffusions, Markov Processes, and Martingales, 2nd edn. Cambridge University Press, Cambridge (2000). ISBN 978-0521775946... 

At least two kinds of advantages come from the use of the semigroup operator approach. First, a mathematically justified approximation of discontinuous processes by continuous processes can be obtained. Second, not only models for purely temporal processes (e.g. pure chemical reactions) can be given, but stochastic models of spatio-temporal phenomena (e.g. chemical reactions with diffusion) can be defined in a mathematically concise manner. The celebrated monograph on the modern approach to Markov processes is still Dynkin s book (1965), and for chemical applications see Arnold Kotelenez (1981). 

What is very important from both the theoretical and practical points of view is the connection between the SDE and the Fokker-Planck equation. If Xq is independent of the solution of (5.150) is a (strong) Markov process, in particular a diffusion process specified by the infinitesimal generator ... 

---