The Kolmogorov differential equation

In the following, we derive the Kolmogorov differential equation on the basis of a simple model and report its various versions. In principle, this equation gives the rate at which a certain state is occupied by the system at a certain time. This equation is of a fundamental importance to obtain models discrete in space and continuous in time. The models, later discussed, are the Poisson Process, the Pure Birth Process, the Polya Process, the Simple Death Process and the Birth-and-Death Process. In section 2.1-3 this equation, i.e. Eq.2-30, has been derived for Markov chains discrete in space and time. 

The movement of inhabitants through an internal city was convincingly demonstrated by Magritte [20] in his painting Golconda depicted in Fig.2-47. 

In setting up the model, we designate by Nj(t) the number of inhabitants occupying state j (an external city j) at time t. Similarly, Nk(t) corresponds to the number of inhabitants occupying state k (an internal city k) at time t. The change in the number of inhabitants in state Sk during time interval At is given by  

ANk(t) = number of inhabitants entering the city in the time interval (t, t + At) 

The quantities on the right-hand side may be expressed by the following probabilities [2, p.59]  

---

A computational method was developed by Gillespie in the 1970s [381, 388] from premises that take explicit account of the fact that the time evolution of a spatially homogeneous process is a discrete, stochastic process instead of a continuous, deterministic process. This computational method, which is referred to as the stochastic simulation algorithm, offers an alternative to the Kolmogorov differential equations that is free of the difficulties mentioned above. The simulation algorithm is based on the reaction probability density function defined below. 

Let Pij i) be the probabihty that a molecule in state i at time zero will be in state j at time t (i,j = 1,2,..., n). It is known that the matrix of transition probabilities P(t) = Py(t) satisfies the Kolmogorov differential equation. Because we can observe the concentration only in the liquid phase of the nth compartment, we are then interested in the probabihty Pi (t), which is the probability that a molecule that is in the first compartment at time zero will move to the nth compartment at time t ... 

From the Kolmogorov equations (4.41) and (4.42), one obtains the difference-differential equations for the birth-death process. The backward equation is given by... 

Besides the hypothesis of spatially homogeneous processes in this stochastic formulation, the particle model introduces a structural heterogeneity in the media through the scarcity of particles when their number is low. In fact, the number of differential equations in the stochastic formulation for the state probability keeps track of all of the particles in the system, and therefore it accounts for the particle scarcity. The presence of several differential equations in the stochastic formulation is at the origin of the uncertainty, or stochastic error, in the process. The deterministic version of the model is unable to deal with the stochastic error, but as stated in Section 9.3.4, that is reduced to zero when the number of particles is very large. Only in this last case can the set of Kolmogorov differential equations be adequately approximated by the deterministic formulation, involving a set of differential equations of fixed size for the states of the process. 

We can overcome this difficulty if instead of using the stochastic differential equations of the process, we use the analysis of the equations with partial derivatives that become characteristic for the passage probabilities (Kolmogorov-type equations). 

Chapter 4 is devoted to the description of stochastic mathematical modelling and the methods used to solve these models such as analytical, asymptotic or numerical methods. The evolution of processes is then analyzed by using different concepts, theories and methods. The concept of Markov chains or of complete connected chains, probability balance, the similarity between the Fokker-Plank-Kolmogorov equation and the property transport equation, and the stochastic differential equation systems are presented as the basic elements of stochastic process modelling. Mathematical models of the application of continuous and discrete polystochastic processes to chemical engineering processes are discussed. They include liquid and gas flow in a column with a mobile packed bed, mechanical stirring of a liquid in a tank, solid motion in a liquid fluidized bed, species movement and transfer in a porous media. Deep bed filtration and heat exchanger dynamics are also analyzed. 

Therefore (95) instead of (60) can be taken as a starting point for what we did in Sect. 6.1. Observe that (95) is the backward Kolmogorov equation associated with the stochastic differential equation (compare (19))... 

Substitution of Eq.(2-112) into Eq.(2-11 la) and approaching At to zero, yields a simplified version of the forward Kolmogorov differential equation for the transition Sj -> Sij-> out of ikth city. This equation is continuous in time and... 

The change in concentration of clusters of n molecules may be written as dCn(t)/dt = an-iCn-i(t) — (ccn + Pn)Cn(t) + Pn+iCn+i(t), which has the form of Kolmogorov differential equation for Markov processes in discrete number space and continuous time [21]. and fin are respectively the net probabilities of incorporation or loss of molecules by a cluster per unit time, and these may be defined formally as the aggregation or detachment frequencies times the surface area of the cluster of n molecules. Given the small size of the clusters, and fin are not simple functions of n and in general they are unknown. However, if and fin are not functions of time, then an equilibrium distribution C° of cluster sizes exists, such that dC°/dt = 0 for Cn t) = C°, and the following differential... 

The developed model of has been mathematically described by a linear differential equation system with constant coefficients. It is therefore assumed that the probability of transitions between states is described by exponential distributions, and, consequently, the intensities of transitions between the states are independent of time. The system of forty-four Chapman-Kolmogorov differential equations have been prepared. 

In order to numerically solve the stochastic differential equation, the constraint that p is constant is made, which is satisfactory so long as the time steps used in the simulation remain sufficiently small during interval t and t + dt. The solutions to Eq. (2.58) has been done by Kolmogorov [13] using different boundary conditions. Due to the extensive use of these Greens functions in the simulation packages, they have been reproduced below using the four most common boundary conditions." ... 

By its random nature, turbulence does not lend itself easily to modelling starting from the differential equations for fluid flow (Navier-Stokes). However, a remarkably successful statistical model due to Kolmogorov has proven very useful for modelling the optical effects of the atmosphere. 

These differential equations depend on the entire probability density function / (x, t) for x(t). The evolution with time of the probability density function can, in principle, be solved with Kolmogorov s forward equation (Jazwinski, 1970), although this equation has been solved only in a few simple cases (Bancha-Reid, 1960). The implementation of practical algorithms for the computation of the estimate and its error covariance requires methods that do not depend on knowing p(x, t). 

Consider a Markov process, which for convenience we take to be homogeneous, so that we may write Tx for the transition probability. The Chapman-Kolmogorov equation (IV.3.2) for Tx is a functional relation, which is not easy to handle in actual applications. The master equation is a more convenient version of the same equation it is a differential equation obtained by going to the limit of vanishing time difference t. For this purpose it is necessary first to ascertain how Tx> behaves as x tends to zero. In the previous section it was found that TX (y2 yl) for small x has the form ... 

The traditional derivation of the Fokker-Planck equation (1.5) or (VIII. 1.1) is based on Kolmogorov s mathematical proof, which assumes infinitely many infinitely small jumps. In nature, however, all jumps are of some finite size. Consequently W is never a differential operator, but always of the type (V.1.1). Usually it also has a suitable expansion parameter and has the canonical form (X.2.3). If it then happens that (1.1) holds, the expansion leads to the nonlinear Fokker-Planck equation (1.5) as the lowest approximation. There is no justification for attributing a more fundamental meaning to Fokker-Planck and Langevin equations than in this approximate sense. 

A one-dimensional Fokker-Planck equation was used by Smoluchowski [19], and the bivariate Fokker-Planck equation in phase space was investigated by Klein [21] and Kramers [22], Note that, in essence, the Rayleigh equation [23] is a monovariate Fokker-Planck equation in velocity space. Physically, the Fokker-Planck equation describes the temporal change of the pdf of a particle subjected to diffusive motion and an external drift, manifest in the second- and first-order spatial derivatives, respectively. Mathematically, it is a linear second-order parabolic partial differential equation, and it is also referred to as a forward Kolmogorov equation. The most comprehensive reference for Fokker-Planck equations is probably Risken s monograph [14]. 

In Appendix 8A we show that when these conditions are satisfied, the Chapman-Kolmogorov integral equation (8.118) leads to two partial differential equations. The Fokker-Planck equation describes the future evolution of the probability distribution... 

The forward equation The Fokker-Planck equation is now derived as the differential form of the Chapman-Kolmogorov equation For any function f (x)... 

For the quantities W(x z,t) equal to zero, the differential Chapman-Kolmogorov equation takes the form of the Fokker-Planck equation ... 

For a stochastic differential equation, there exists an associated Fokker - Planck equation, which describes the probability that the variable takes the value concerned. The Fokker - Planck equation is also called the forward Kolmogorov equation. To the particular stochastic differential equation (21.13) the following Fokker - Planck equation is associated ... 

In this section we remind the reader of the Kolmogorov forward and backward equations, infinitesimal generators, stochastic differential equations, and functional integrals and then consider how the basic transport equations are related to underlying Markov stochastic processes [141, 142],... 

System presented on the figure 4 may be described by the following differential Kolmogorov-Chapman equations ... 

Using assumption (4) and adopting the condition that the process is homogeneous in time (i.e transition probability function depends only on t - s and not specifically on / and s) the Kolmogorov equations (5.9) and (5.10) can be reduced to (linear) ordinary differential-difference equation ( differential in time, and difference in state and in accordance with (5.27) we get, for a deterministic initial condition... 

The derivation of a differential equation for p(r, v, t) is performed by first defining the diffusion process as an independent Markov process to write a Chapman-Kolmogorov equation in phase space ... 

Let P x, t) be the matrix of transition probabilities Pj,(r, /). It can be shown that the transition probabilities satisfy the Kolmogorov forward differential equations that is,... 

The transition probabihty column vector P(t) = Py(t), j = 1,2, 3, satisfies the Kolmogorov forward differential equation [3] ... 

---