Markovian chains

Chvosta et alP consider another case where the work probability distribution function can be determined. They study a two energy-level system, modelled as a stochastic, Markovian process, where the transition rates and energies depend on time. Like the previous examples it provides an exact model that can be used to assist in identifying the accuracy of approximate, numerical studies. Ge and Qian extended the stochastic derivation for a Markovian chain to a inhomogeneous Markov chain. 

Statistics of Copolymer Chains. A triad distribution is capable of giving even more microstructural information after comparing an experimental distribution to a calculated distribution having specifically defined chain statistics (3,6,15,16). The calculated statistical distributions can be random or some higher level of a so-called Markovian chain. These models are constructed on the premise of a conditional probability of finding any particular unit or sequence in a copolymer chain. In a perfectly random statistical model, there are no previous conditions required for one unit to follow any other particular unit. These types of statistical models are called Bernoullian or zero-order Markovian. For these statistical models, only mole fraction information is required to determine what the corresponding triad distribution would look like for a perfectly random copolymer, that is. 

An excellent way to treat such data is to use reaction probability models.(1,2) In the NMR analysis of tacticity, it is frequently possible to distinguish whether the configuration is chain-end controlled or catalytic-site controlled during polymerization. Various statistical models have been proposed. The chain-end controlled models include Bemoullian (B), and first- and second-order Markovian (Ml and M2) statistics.(1) The simplest catalytic-site controlled model is the enantiomorphic site (E) model.(3) The relationship between the chain-end and catalytic-site controlled models and possible hybrid models have been delineated in a recent article.(4)... 

It should be emphasized that for Markovian copolymers a knowledge of the values of structural parameters of such a kind will suffice to find the probability of any sequence Uk, i.e. for an exhaustive description of the microstructure of the chains of these copolymers with a given average composition. As for the composition distribution of Markovian copolymers, this obeys for any fraction of Z-mers the Gaussian formula whose covariance matrix elements are Dap/l where Dap depend solely on the values of structural parameters [2]. The calculation of their dependence on time, and the stoichiometric and kinetic parameters of the reaction system permits a complete statistical description of the chemical structure of Markovian copolymers to be accomplished. The above reasoning reveals to which extent the mathematical modeling of the processes of the copolymer synthesis is easier to perform provided the alternation of units in macromolecules is known to obey Markovian statistics. 

The instantaneous composition of a copolymer X formed at a monomer mixture composition x coincides, provided the ideal model is applicable, with stationary vector ji of matrix Q with the elements (8). The mathematical apparatus of the theory of Markov chains permits immediately one to wright out of the expression for the probability of any sequence P Uk in macromolecules formed at given x. This provides an exhaustive solution to the problem of sequence distribution for copolymers synthesized at initial conversions p l when the monomer mixture composition x has had no time to deviate noticeably from its initial value x°. As for the high-conversion copolymerization products they evidently represent a mixture of Markovian copolymers prepared at different times, i.e. under different concentrations of monomers in the reaction system. Consequently, in order to calculate the probability of a certain sequence Uk, it is necessary to average its instantaneous value P Uk over all conversions p preceding the conversion p up to which the synthesis was conducted. 

This is the simplest of the models where violation of the Flory principle is permitted. The assumption behind this model stipulates that the reactivity of a polymer radical is predetermined by the type of bothjts ultimate and penultimate units [23]. Here, the pairs of terminal units MaM act, along with monomers M, as kinetically independent elements, so that there are m3 constants of the rate of elementary reactions of chain propagation ka ]r The stochastic process of conventional movement along macromolecules formed at fixed x will be Markovian, provided that monomeric units are differentiated by the type of preceding unit. In this case the number of transient states Sa of the extended Markov chain is m2 in accordance with the number of pairs of monomeric units. No special problems presents writing down the elements of the matrix of the transitions Q of such a chain [ 1,10,34,39] and deriving by means of the mathematical apparatus of the Markov chains the expressions for the instantaneous statistical characteristics of copolymers. By way of illustration this matrix will be presented for the case of binary copolymerization ... 

An exhaustive statistical description of living copolymers is provided in the literature [25]. There, proceeding from kinetic equations of the ideal model, the type of stochastic process which describes the probability measure on the set of macromolecules has been rigorously established. To the state Sa(x) of this process monomeric unit Ma corresponds formed at the instant r by addition of monomer Ma to the macroradical. To the statistical ensemble of macromolecules marked by the label x there corresponds a Markovian stochastic process with discrete time but with the set of transient states Sa(x) constituting continuum. Here the fundamental distinction from the Markov chain (where the number of states is discrete) is quite evident. The role of the probability transition matrix in characterizing this chain is now played by the integral operator kernel ... 

As the result of theoretical consideration of polycondensation of an arbitrary mixture of such monomers it was proved [55,56] that the alternation of monomeric units along polymer molecules obey the Markovian statistics. If all initial monomers are symmetric, i.e. they resemble AaScrAa, units Sa(a=l,...,m) will correspond to the transient states of the Markov chain. The probability vap of transition from state Sa to is the ratio Q /v of two quantities Qa/9 and va which represent, respectively, the number of dyads (SaSp) and monads (Sa) per one monomeric unit. Clearly, Qa(S is merely a ratio of the concentration of chemical bonds of the u/i-ih type, formed as a result of the reaction between group Aa and Ap, to the overall concentration of monomeric units. The probability va0 of a transition from the transient state Sa to an absorbing state S0 equals l-pa where pa represents the conversion of groups Aa. 

For the interbipolycondensation the condition of quasiideality is the independence of the functional groups either in the intercomponent or in both comonomers. In the first case the sequence distribution in macromolecules will be described by the Bernoulli statistics [64] whereas, in the second case, the distribution will be characterized by a Markov chain. The latter result, as well as the parameters of the above mentioned chain, were firstly obtained within the framework of the simplified kinetic model [64] and later for its complete version [59]. If all three monomers involved in interbipolycondensation have dependent groups then, under a nonequilibrium regime, non-Markovian copolymers are known to form. 

Thus, as can be inferred from the foregoing, the calculation of any statistical characteristics of the chemical structure of Markovian copolymers is rather easy to perform. The methods of statistical chemistry [1,3] can reveal the conditions for obtaining a copolymer under which the sequence distribution in macromolecules will be describable by a Markov chain as well as to establish the dependence of elements vap of transition matrix Q of this chain on the kinetic and stoichiometric parameters of a reaction system. It has been rigorously proved [ 1,3] that Markovian copolymers are formed in such reaction systems where the Flory principle can be applied for the description of macromolecular reactions. According to this fundamental principle, the reactivity of a reactive center in a polymer molecule is believed to be independent of its configuration as well as of the location of this center inside a macromolecule. 

Eigenvalues of the operator Qr are real while the largest of them, Af, equals unity by definition. As a result, in the limit n-> oo all items in the sum (Eq. 38), excluding the first one, Q Q f = Xr/Xfh will vanish. In this case, chemical correlators will decay exponentially along the chain on the scale n 1/ In AAt values n < n the law of the decay of these correlators differs, however, from the exponential one even for binary copolymers. This obviously testifies to non-Markovian statistics of the sequence distribution in molecules (see expression Eq. 11). The closer is to unity, the greater are the values of n. The situation when n 1 corresponds to proteinlike copolymers. 

For many synthetic copolymers, it becomes possible to calculate all desired statistical characteristics of their primary structure, provided the sequence is described by a Markov chain. Although stochastic process 31 in the case of proteinlike copolymers is not a Markov chain, an exhaustive statistic description of their chemical structure can be performed by means of an auxiliary stochastic process 3iib whose states correspond to labeled monomeric units. As a label for unit M , it was suggested [23] to use its distance r from the center of the globule. The state of this stationary stochastic process 31 is a pair of numbers, (a, r), the first of which belongs to a discrete set while the second one corresponds to a continuous set. Stochastic process ib is remarkable for being stationary and Markovian. The probability of the transition from state a, r ) to state (/i, r") for the process of conventional movement along a heteropolymer macromolecule is described by the matrix-function of transition intensities... 

The Markovian approach is twofold. If the electrolyzer is viewed as a Markov processor, Eqs. (27) and (28) are replaced by their Kolmogorov-equivalents, Eqs. (8) and (9) the solution is obtained in terms of probabilities for state, V (species concentration A), and state. s 2 (species concentration 8). It is more inspiring (elegant ) to regard the electrolyzer as a Markov-chain generator with Markov equations... 

In Section 9.1.1 we have introduced a stochastic model for the description of surface reaction systems which takes correlations explicitly into account but neglects the energetic interactions between the adsorbed particles as well as between a particle and a metal surface. We have formulated this by master equations upon the assumption that the systems are of the Markovian type. In the model an infinite set of master equations for the distribution functions of the state of the surface and of pairs of surface sites (and so on) arise. This chain of equations cannot be solved analytically. To handle this problem practically this hierarchy was truncated at a certain level. The resulting equations can be solved numerically exactly in a small region and can be connected to a mean-field solution for large distances from a reference point. 

When one is interested in slow modes of motion of the system, each macromolecule of the system can be schematically described in a coarse-grained way as consisting of N + 1 linearly-coupled Brownian particles, and we shall be able to look at the system as a suspension of n(N + 1) interacting Brownian particles. An anticipated result for dynamic equation of the chains in equilibrium situation can be presented as a system of stochastic non-Markovian equations... 

---