Markov connection

As far as the behaviour of the particle in such a system respects the rules of the Markov process, it will be controlled by a Markov connection. This means that the probability of the particle occurrence within the k cell after n + 1 time (i.e. t = n.At) is given only by its probability of occurrence in the j cell after time n and by its probability of transfer from the j cell to the k cell denoted pj. Now we can write ... 

If the parameters a j have constant values, then the model described by system (4.71) corresponds to a Markov connection linking the process components. In this case, as in general, the process components represent the individual displacements which can be characterized globally through the convective mixing of their spectra of speeds ( v, k= 1,N). 

We begin the discussion by referring to the stochastic model given by relation (4.58), which is rewritten here as shown in relation (4.120). Here for a finite Markov connection process we must consider the constant time values for all the elements of the matrix P = pij i k-... 

Some restrictions are imposed when we start the application of limit theorems to the transformation of a stochastic model into its asymptotic form. The most important restriction is given by the rule where the past and future of the stochastic processes are mixed. In this rule it is considered that the probability that a fact or event C occurs will depend on the difference between the current process (P(C) = P(X(t)e A/V(X(t))) and the preceding process (P (C/e)). Indeed, if, for the values of the group (x,e), we compute = max[P (C/e) — P(C)], then we have a measure of the influence of the process history on the future of the process evolution. Here, t defines the beginning of a new random process evolution and tIt- gives the combination between the past and the future of the investigated process. If a Markov connection process is homogenous with respect to time, we have = 1 or Tt O after an exponential evolution. If Tt O when t increases, the influence of the history on the process evolution decreases rapidly and then we can apply the first type limit theorems to transform the model into an asymptotic... 

If we consider a process where the elementary states Vj, V2,. v work with a Markov connection, this connection presents an associated generator of probabil-... 

For our considered process, where the states Vj, V2,. Vj.j are Markov connected, the variable term for this last relation (f v(a)da j tends [4.5] towards a normal... 

Very difficult problems occur with the asymptotic transformation of original stochastic models based on stochastic differential equations where the elementary states are not Markov connected. This fact will be discussed later in this chapter (for instance see the discussion of Eq. (4.180)). 

The algorithm to compute a stochastic model with two Markov connected elementary states is shown in Fig. 4.11. Here, the process state evolves with constant Vj and V2 speeds. This model is a particularization of the model commented above (see the assembly of relations (4.146)-(4.147)) and has the following mathematical expression ... 

For our stochastic process, the probabilities Pij,i5 j result from the Markov connections which are described as follows ... 

The connection between a diffusion equation and a corresponding Markov diffusion process may be established through expressions for drift velocities and diffusitivies. The drift velocity for both unconstrained and constrained systems may be expressed in an arbitrary system of coordinates in the generic form... 

Once a connection is made it is discontinued at a random moment, rn = an. (This somewhat unrealistic assumption is needed for the Markov character.) The master equation is... 

In the case of a photoconductor p is increased by a constant y proportional to the incident light intensity. The system is no longer closed and the new P is no longer connected with a by detailed balance. The stationary solution (9.2) is no longer identical with the thermodynamic equilibrium. Another remark is, that it is possible to represent the effect of the incident photons in this simple way of adding y to the generation probability only if the arrival times of the photons are uncorrelated (shot noise). When they are correlated the number n is no longer a Markov process, and a more sophisticated description is needed, see XV.3. 

For instance, the components Xj(i = 1,2) of the composition vector X according to determination (3.16) are connected with components Jij(i = 1,2,3,4) of the stationary vector of the auxiliary Markov chain in the same manner (3.13) as in the penultimate model. However, the values of Aj in expressions (3.8) are determined, instead of formula (3.11), by the following expressions ... 

This statement can also be obtained when a transport process evolution is analyzed by the concept of Markov chains or completely connected chains. The math-... 

Some of the examples shown in the following paragraphs present the characteristics of a random system with complete connections. However, other examples do not concern a completely connected system but present only some Markov unitary processes [4.6, 4.17]. 

The establishment of stochastic equations frequently results from the evolution of the analyzed process. In this case, it is necessary to make a local balance (space and time) for the probability of existence of a process state. This balance is similar to the balance of one property. It means that the probability that one event occurs can be considered as a kind of property. Some specific rules come from the fact that the field of existence, the domains of values and the calculation rules for the probability of the individual states of processes are placed together in one or more systems with complete connections or in Markov chains. 

Polystochastic models are used to characterize processes with numerous elementary states. The examples mentioned in the previous section have already shown that, in the establishment of a stochastic model, the strategy starts with identifying the random chains (Markov chains) or the systems with complete connections which provide the necessary basis for the process to evolve. The mathematical description can be made in different forms such as (i) a probability balance, (ii) by modelling the random evolution, (iii) by using models based on the stochastic differential equations, (iv) by deterministic models of the process where the parameters also come from a stochastic base because the random chains are present in the process evolution. 

Model (4.79) describes an evolutionary process, which results from the coupling of a Markov chain assistance with some individual diffusion processes. This model is well known in the study of the coupling of a chemical reaction with diffusion phenomena [4.5, 4.6, 4.34, 4.35]. The models described by relations (4.63) and (4.79) can still be particularized or generalized. As an example, we can notice that other types of models can be suggested if we consider that the values of Ukj are functions of z or t or Pk(z,r) in Eq.(4.79). However, it is important to observe that the properties of the Markov type connections cannot be considered when kj =f(PK(z. c))-... 

Both equations give good results for the description of mass and heat transport without forced flow. Here, it is important to notice that the Fokker-Plank-Kolmo-gorov equation corresponds to a Markov process for a stochastic connection. Consequently, it can be observed as a solution to the stochastic equations written below ... 

By introducing the stochastic Markov type connection process through the following equation ... 

The stochastic model accepts a Markov type connection between both elementary states. So, with ai2Ar, we define the transition probability from type I to type II, whereas the transition probability from type II to a type I is a2iAr. By Pi(x,t) and P2(x, t) we note the probability of locating the microparticle at position x and time T with a type I or respectively a type II evolution. With these introductions and notations, the general stochastic model (4.71) gives the particularization written here by the following differential equation system ... 

Chapter 4 is devoted to the description of stochastic mathematical modelling and the methods used to solve these models such as analytical, asymptotic or numerical methods. The evolution of processes is then analyzed by using different concepts, theories and methods. The concept of Markov chains or of complete connected chains, probability balance, the similarity between the Fokker-Plank-Kolmogorov equation and the property transport equation, and the stochastic differential equation systems are presented as the basic elements of stochastic process modelling. Mathematical models of the application of continuous and discrete polystochastic processes to chemical engineering processes are discussed. They include liquid and gas flow in a column with a mobile packed bed, mechanical stirring of a liquid in a tank, solid motion in a liquid fluidized bed, species movement and transfer in a porous media. Deep bed filtration and heat exchanger dynamics are also analyzed. 

Transition networks have also been constructed for biomolecules directly from MD data, - where they are sometimes referred to as Markov state models. In order to gather sufficient statistics, the transitions between states must be observable on an MD timescale. Alternative methods exist, not based on MD, where connections between conformations are inferred based on distance criteria. The edge weights of the corresponding graphs are then based on the energies of the two geometries that are assumed to be connected, rather than calculated barriers or rate constants. 

---