Non-Markov processes

In this review we show that there are two main sources of memory. One of them correspond to the memory responsible for Anderson localization, and it might become incompatible with a representation in terms of trajectories. The fluctuation-dissipation process used here to illustrate Anderson localization in the case of extremely large Anderson randomness is an idealized condition that might not work in the case of correlated Anderson noise. On the other hand, the non-Poisson renewal processes generate memory properties that may not be reproduced by the stationary correlation functions involved by the projection approach to the GME. Before ending this subsection, let us limit ourselves to anticipating the fundamental conclusion of this review The CTRW is a correct theoretical tool to address the study of the non-Markov processes, if these correspond to trajectories undergoing unpredictable jumps. 

Example 2.4 demonstrates a non-Markov process where the past history must be taken into account for prediction of the future. We consider the state of Israel as the system which has undergone many wars during the last fifty years. This situation is demostrated schematically as follows ... 

One should be careful with this procedure, as in principle it renders a Monte Carlo simulation a non-Markov process. The effect is likely to be benign, but the safest way to proceed is to take the corrector updates of the pressure only during the equilibration phase of the simulation (i.e., those cycles normally granted to allow the system to equilibrate to the new state conditions). In our experience the corrector iteration usually converges very quickly, well before the end of the equilibration period. As a check one can... 

Formally, evolution equation for non-Markov processes can be described as integro-differential equations ... 

Medina-Noyola, M. and del Rfo-Correa, J. L. 1987. The fluctuation dissipation theorem for non-Markov process and their contractions The role of the stationary condition. Physica A 146 483. 

While the Smoliichowski equation is necessary for a Markov process, in general it is not sufficient, but known counter-examples are always non-Gaiissian as well. 

The key quantity in barrier crossing processes in tiiis respect is the barrier curvature Mg which sets the time window for possible influences of the dynamic solvent response. A sharp barrier entails short barrier passage times during which the memory of the solvent environment may be partially maintained. This non-Markov situation may be expressed by a generalized Langevin equation including a time-dependent friction kernel y(t) [ ]... 

More complex schemes have been proposed, such as second-order Markov chains with four independent parameters (corresponding to a copolymerization with penultimate effect, that is, an effect of the penultimate member of the growing chain), the nonsymmetric Bernoulli or Markov chains, or even non-Maikov models a few of these will be examined in a later section. Verification of these models calls for the knowledge of the distribution of sequences that become longer, the more complex the proposed mechanism. Considering only Bernoulli and Markov processes it may be said that at the dyad level all models fit the experimental data and hence none can be verified at the triad level the Bernoulli process can be verified or rejected, while all Markov processes fit at the tetrad level the validity of a first-order Markov chain can be confirmed, at the pentad level that of a second-order Maikov chain, and so on (10). 

If one chooses Pi(Vi, 0) = fi ) a non-stationary Markov process is defined, called the Wiener process or Wiener-Levy process. ) It is usually considered for f >0 alone and was originally invented for describing the stochastic behavior of the position of a Brownian particle (see VIII.3). The probability density for t > 0 is according to (2.2)... 

Even when a system is in a steady state other than equilibrium certain physical quantities may be stationary Markov processes. An example are the current fluctuations in the circuit of fig. 7 when a battery is added, which maintains a constant potential difference and therefore a non-zero average current. Another example is a Brownian particle in a homogeneous gravitational field its vertical velocity is a stationary process, but not its position. 

Find the Pn for this non-Gaussian stationary Markov process. 

These processes are non-stationary because the condition singled out a certain time t0. Yet their transition probability depends on the time interval alone as it is the same as the transition probability of the underlying stationary process. Non-stationary Markov processes whose transition probability depends on the time difference alone are called homogeneous processes. 10 They usually occur as subensembles of stationary Markov processes in the way described here. However, the Wiener process defined in 2 is an example of a homogeneous process that cannot be embedded in a stationary Markov process. 

Random walks on square lattices with two or more dimensions are somewhat more complicated than in one dimension, but not essentially more difficult. One easily finds, for instance, that the mean square distance after r steps is again proportional to r. However, in several dimensions it is also possible to formulate the excluded volume problem, which is the random walk with the additional stipulation that no lattice point can be occupied more than once. This model is used as a simplified description of a polymer each carbon atom can have any position in space, given only the fixed length of the links and the fact that no two carbon atoms can overlap. This problem has been the subject of extensive approximate, numerical, and asymptotic studies. They indicate that the mean square distance between the end points of a polymer of r links is proportional to r6/5 for large r. A fully satisfactory solution of the problem, however, has not been found. The difficulty is that the model is essentially non-Markovian the probability distribution of the position of the next carbon atom depends not only on the previous one or two, but on all previous positions. It can formally be treated as a Markov process by adding an infinity of variables to take the whole history into account, but that does not help in solving the problem. 

Exercise. The solution of the ordinary M-equation for non-composite Markov processes, such as (V.1.5), can also be written as a sum over realizations. The result is, in analogy with (7.10),... 

The stochastic function X(t) by itself is not Markovian. This is an example of the fact discussed in IV. 1 If one has an r-component Markov process and one ignores some of the components, the remaining sstochastic process but in general not Markovian. Conversely, it is often possible to study non-Marko-vian processes by regarding them as the projection of a Markov process with more components. We return to this point in IX.7. 

Therefore, the simplest procedure to get the stochastic description of the reaction leads to the rather complicated set of equations containing phenomenological parameters / (equation (2.2.17)) with non-transparent physical meaning. Fluctuations are still considered as a result of the external perturbation. An advantage of this approach is a useful analogy of reaction kinetics and the physics of equilibrium critical phenomena. As is well known, because of their nonlinearity, equations (2.1.40) reveal non-equilibrium bifurcations [78, 113]. A description of diffusion-controlled reactions in terms of continuous Markov process - equation (2.2.15) - makes our problem very similar to the static and dynamic theory of critical phenomena [63, 87]. When approaching the bifurcation points, the systems with reactions become very sensitive to the environment fluctuations, which can even produce new nonequilibrium transitions [18, 67, 68, 90, 108]. The language developed in the physics of critical phenomena can be directly applied to the processes in spatially extended systems. 

In the past, the equivalence between the size distribution generated by the Smoluchowski equation and simple statistical methods [9, 12, 40-42] was a source of some confusion. The Spouge proof and the numerical results obtained for the kinetics models with more complex aggregation physics, e.g., with a presence of substitution effects [43,44], revealed the non-equivalence of kinetics and statistical models of polymerization processes. More elaborated statistical models, however, with the complete analysis made repeatedly at small time intervals have been shown to produce polymer size distributions equivalent to those generated kinetically [45]. Recently, Faliagas [46] has demonstrated that the kinetics and statistical models which are both the mean-field models can be considered as special cases of a general stochastic Markov process. 

The Need for Generalization of the Kramers Theory The Generalized Kramers Model Non-Markovian Effects in the One-Dimensional Case The Escape Rate of a Non-Markov Multidimensional Process... 

If 21(f) is a Markov process with continuous transition probabilities and T(t) a process with non-negative independent increments, then X(T(t)) is also a Markov process. Thus, this process is subordinated to 21(f) with operational time T(t). The process 7 (f) is called a directing (controlling) process. 

An example of a simple but non-trivial system is presented in (Bouissou Dutuit 2004). There, it is modeled in two different ways by a stochastic Petri net and by a Boolean Driven Markov Process (BDMP). In the following, we will present a model of the same system using LARES. 

A faster numerical procedure for solving non-homogeneous semi-Markov processes... 

Continuous time non-homogeneous semi-Markov processes (CTNHSMP) are powerful modeling tools, especially in the reliability field (as exemplified in Janssen Manca (2007)). According to Becker et al. (2000), CTNHSMP are considered as approaches to model reliability characteristics of components or small systems with complex test and maintenance strategies. 

The semi-Markov process is therefore considered as non-homogeneous so that this deterioration process may be adequately addressed. Therefore, the required data to estimate the system availability via this CTNHSMP model are the parameters pyi ) and... 

To conclude, two important limitations of this work deserve attention. Firstly discussing semi-Markov processes in general, we have the well-known and already cited difficulty in obtaining the requisite data to analyze semi-Markov processes on the non-homogeneous environment. On that, El-Gohary (2004) presents max-iminn likelihood and Bayes estimates of the parameters included in a semi-Markov reliability model of three states. 

---