Stochastic processes time evolution

As discussed in Section 8.2.1, the Langevin equation (8.13) describes a Markovian stochastic process The evolution of the stochastic system variable x(Z) is determined by the state of the system and the bath at the same time t. The instantaneous response of the bath is expressed by the appearance of a constant damping coefficient y and by the white-noise character of the random force 7 (Z). 

The stochastic problem is to describe properly the time evolution of the Heisenberg operator d(t) averaged over all the realizations of collisional process in the interval (0,t). The averaging, performed in the impact theory, results in the phenomenological kinetic equation [170, 158]... 

The aim of this chapter is to describe approaches of obtaining exact time characteristics of diffusion stochastic processes (Markov processes) that are in fact a generalization of FPT approach and are based on the definition of characteristic timescale of evolution of an observable as integral relaxation time [5,6,30—41]. These approaches allow us to express the required timescales and to obtain almost exactly the evolution of probability and averages of stochastic processes in really wide range of parameters. We will not present the comparison of these methods because all of them lead to the same result due to the utilization of the same basic definition of the characteristic timescales, but we will describe these approaches in detail and outline their advantages in comparison with the FPT approach. 

Consider, on the other hand, a purely stochastic, Markovian process. The evolution of a dynamical function /(oa) is now determined by a transition probability / (co, r co, 0) from the state oa at time zero, to the state ca at time t > 0 This gives rise to a transformation Wt off ... 

Third, Eq. (31) shows that A is nondistributive, and determines fluctuations. Since there is a flucmation, we can expect that the time evolution in Eq. (34) may be related to a stochastic process. Indeed, one can show that the time evolution (34) is identical to the time evolution generated by the set of Langevin equations for the stochastic operators aj(r), a (r) (see Ref. 14) ... 

A computational method was developed by Gillespie in the 1970s [381, 388] from premises that take explicit account of the fact that the time evolution of a spatially homogeneous process is a discrete, stochastic process instead of a continuous, deterministic process. This computational method, which is referred to as the stochastic simulation algorithm, offers an alternative to the Kolmogorov differential equations that is free of the difficulties mentioned above. The simulation algorithm is based on the reaction probability density function defined below. 

By carrying out the above procedure from time 0 to time /,Mm, we evidently obtain only one possible realization of the stochastic process. In order to get a statistically complete picture of the temporal evolution of the system, we must actually carry out several independent realizations or runs. These runs must use the same initial conditions of the problem but different starting numbers for the uniform random number generator in order for the algorithm to result in different but statistically equivalent chains. If we make K runs in all, and record the population sizes (k, t) in run k at time t (i = 1,..., m and k = 1,...,K), then we may assert that the average number of particles at time t is... 

At the same time, it is known that, during exploitation of stochastic models, cases that show great difficulty concerning the selection and the choice of some parameters of the models frequently appear. As a consequence, the original models become unattractive for research by simulation. In these cases, the models can be transformed to equivalent models which are distorted but exploitable. The use of stochastic distorted models is also recommended for the models based on stochastic chains or polystocastic processes where an asymptotic behaviour is identified with respect to a process transition matrix of probabilities, process chains evolution, process states connection, etc. The distorted models are also of interest when the stochastic process is not time dependent, as, for example, in the stochastic movement of a marked particle occurring with a constant velocity vector, like in diffusion processes. 

For the derivation of one asymptotic variant of a given polystochastic model of a process, we can use the perturbation method. For this transformation, a new time variable is introduced into the stochastic model and then we analyze its behaviour. The new time variable is t = eT, which includes the time evolution t and an arbitrary parameter e, which allows the observation of the model behaviour when its values become very small (e—>0). Here, we study the changes in the operator 0(t, t) when e 0 whilst paying attention to having stable values for t/e or t/e. ... 

Some restrictions are imposed when we start the application of limit theorems to the transformation of a stochastic model into its asymptotic form. The most important restriction is given by the rule where the past and future of the stochastic processes are mixed. In this rule it is considered that the probability that a fact or event C occurs will depend on the difference between the current process (P(C) = P(X(t)e A/V(X(t))) and the preceding process (P (C/e)). Indeed, if, for the values of the group (x,e), we compute = max[P (C/e) — P(C)], then we have a measure of the influence of the process history on the future of the process evolution. Here, t defines the beginning of a new random process evolution and tIt- gives the combination between the past and the future of the investigated process. If a Markov connection process is homogenous with respect to time, we have = 1 or Tt O after an exponential evolution. If Tt O when t increases, the influence of the history on the process evolution decreases rapidly and then we can apply the first type limit theorems to transform the model into an asymptotic... 

If we consider the evolution of the liquid element together with the state of probabilities of elementary evolutions, we can observe that we have a continuous Markov stochastic process. If we apply the model given in Eq. (4.68), Pj(z, t) is the probability of having the liquid element at position x and time t evolving by means of a type 1 elementary process (displacement with a d-v flow rate along a positive direction of x). This probability can be described through three independent events ... 

Several attempts to describe replication-mutation networks by stochastic techniques were made in the past. We cannot discuss them in detail here, but we shall brieffy review some general ideas that are relevant for the quasispecies model. The approach that is related closest to our model has been mentioned already [51] the evolutionary process is viewed as a sequence of stepwise increases in the populations mean fitness. Fairly long, quasi-stationary phases are interrupted by short periods of active selection during which the mean fitness increases. The approach towards optimal adaptation to the environment is resolved in a manner that is hierarchical in time. Evolution taking place on the slow time scale represents optimization in the whole of the sequence space. It is broken up into short periods of time within which the quasi-species model applies only locally. During a single evolutionary step only a small part of sequence space is explored by the population. There, the actual distributions of sequences resemble local quasispecies confined to well-defined regions. Error thresholds can be defined locally as well. 

Photoinduced charge separation processes in the supramolecular triad systems D -A-A, D -A -A and D -A-A have been investigated using three potential energy surfaces and two reaction coordinates by the stochastic Liouville equation to describe their time evolution. A comparison has l n made between the predictions of this model and results involving charge separation obtained experimentally from bacterial photosynthetic reaction centres. Nitrite anion has been photoreduced to ammonia in aqueous media using [Ni(teta)] " and [Ru(bpy)3] adsorbed on a Nafion membrane. 

Time correlation functions. If we look at 5p(r, f at a given r as a function of time, its time evolution is an example of a stochastic process (see Chapter 7). In a given time Zi the variables p(r, fi) and p(r, ti) = p (r, fi) —p are random variables in the sense that repeated measurements done on different identical systems will give different realizations for these variables about the average p. Again, for the random variables x = Sp(r,tf andy = 8p r,t") wq can look at the correlation... 

We can measure and discuss z(Z) directly, keeping in mind that we will obtain different realizations (stochastic trajectories) of this function from different experiments performed imder identical conditions. Alternatively, we can characterize the process using the probability distributions associated with it. P(z, Z)random variable z at time Z is in the interval between z and z +- dz. P2(z2t2 zi fi )dzidz2 is the probability that z will have a value between zi and zi + dz at Zi and between Z2 and Z2 -F t/z2 at t, etc. The time evolution of the process, if recorded in times Zo, Zi, Z2, - - , Zn is most generally represented by the joint probability distribution Piz t , , z iUp. Note that any such joint distribution function can be expressed as a reduced higher-order function, for example. 

The time evolution in a Markovian stochastic process is therefore fully described by the transition probability Pilyt z T). 

What is the significance of the Markovian property of a physical process Note that the Newton equations of motion as well as the time-dependent Schrodinger equation are Markovian in the sense that the future evolution of a system described by these equations is fully determined by the present ( initial ) state of the system. Non-Markovian dynamics results from reduction procedures used in order to focus on a relevant subsystem as discussed in Section 7.2, the same procedures that led us to consider stochastic time evolution. To see this consider a universe described by two variables, zi and z, which satisfy the Markovian equations of motion... 

The time evolution of stochastic processes can be described in two ways ... 

We have already noted the difference between the Langevin description of stochastic processes in terms of the stochastic variables, and the master or Fokker-Planck equations that focus on their probabilities. Still, these descriptions are equivalent to each other when applied to the same process and variables. It should be possible to extract information on the dynamics of stochastic variables from the time evolution of their probabihty distribution, for example, the Fokker-Planck equation. Here we show that this is indeed so by addressing the passage time distribution associated with a given stochastic process. In particular we will see (problem 14.3) that the first moment of this distribution, the mean first passage time, is very useful for calculating rates. 

Passage time distributions and the mean first passage time provide a useful way for analyzing the time evolution of stochastic processes. An application to chemical reactions dominated by barrier crossing is given in Section 14.4.2 and Problem 14.3. 

Not only is the probability of discharge d of interest, but also the current evolution with time. From the description of the current evolution with time one can deduce the mean current, which is easily accessible experimentally and is important to evaluate the quantity of heat generated by the process. From the description of the stochastic process of the discharge activity given previously, the current can be computed. In order to obtain the current evolution equation, an auxiliary random variable n(t) defined by the time derivation of the random variable N(t) is introduced ... 

---