Markov chain/process

The Metropolis-Hastings algorithm is the most general form of the MCMC processes. It is also the easiest to conceptualize and code. An example of pseudocode is given in the five-step process below. The Markov chain process is clearly shown in the code, where samples that are generated from the prior distribution are accepted as arising from the posterior distribution at the ratio of the probability of the joint... 

Here, the conformational space is sampled by a set of MC moves through a Markov chain process. A trial move from a conformation (or... 

The rank of the square matrix P is one greater than the number of transient states in the Markov chain process for the polymerization. To analyse step-growth polymerizations the most convenient approach is to regard the sequential counting process on already formed chains, for a given extent of reaction, as being a Markov process. A particular transient state is then identified as the presence of a particular comonomer unit at some position along the chain, and the transition probabilities between the transient states... 

M. D. Sawhney, A study of ocean wave amplitudes in terms of the theory of runs and a Markov chain process. Tech. Rep. of New York University (1962), p. 29. 

We, along with others (12,13,14), have considered a model by which the stereosequence distribution, i.e., distribution of triad sequences along the polymer chain, is described as a Markov chain process (15). Addition of a new repeating unit to a particular triad leads to a new triad consisting of the last two units plus the unit added. The transition probabilities associated with the various possibilities that can thus arise from the four triads can be defined and are given in Table II. 

In NOWIcob weather time series are generated by applying a Markov chain process and historical weather data are used to estimate transition matrices from one weather state to the next one (Hofmann and Sperstad, 2013). 

In the framework of this ultimate model [33] there are m2 constants of the rate of the chain propagation kap describing the addition of monomer to the radical Ra whose reactivity is controlled solely by the type a of its terminal unit. Elementary reactions of chain termination due to chemical interaction of radicals Ra and R is characterized by m2 kinetic parameters k f . The stochastic process describing macromolecules, formed at any moment in time t, is a Markov chain with transition matrix whose elements are expressed through the concentrations Ra and Ma of radicals and monomers at this particular moment in the following way [1,34] ... 

This is the simplest of the models where violation of the Flory principle is permitted. The assumption behind this model stipulates that the reactivity of a polymer radical is predetermined by the type of bothjts ultimate and penultimate units [23]. Here, the pairs of terminal units MaM act, along with monomers M, as kinetically independent elements, so that there are m3 constants of the rate of elementary reactions of chain propagation ka ]r The stochastic process of conventional movement along macromolecules formed at fixed x will be Markovian, provided that monomeric units are differentiated by the type of preceding unit. In this case the number of transient states Sa of the extended Markov chain is m2 in accordance with the number of pairs of monomeric units. No special problems presents writing down the elements of the matrix of the transitions Q of such a chain [ 1,10,34,39] and deriving by means of the mathematical apparatus of the Markov chains the expressions for the instantaneous statistical characteristics of copolymers. By way of illustration this matrix will be presented for the case of binary copolymerization ... 

In order to obtain the expression for the components of the vector of instantaneous copolymer composition it is necessary, according to general algorithm, to firstly determine the stationary vector ji of the extended Markov chain with the matrix of transitions (13) which describes the stochastic process of conventional movement along macromolecules with labeled units and then to erase the labels. In this particular case such a procedure reduces to the summation ... 

An exhaustive statistical description of living copolymers is provided in the literature [25]. There, proceeding from kinetic equations of the ideal model, the type of stochastic process which describes the probability measure on the set of macromolecules has been rigorously established. To the state Sa(x) of this process monomeric unit Ma corresponds formed at the instant r by addition of monomer Ma to the macroradical. To the statistical ensemble of macromolecules marked by the label x there corresponds a Markovian stochastic process with discrete time but with the set of transient states Sa(x) constituting continuum. Here the fundamental distinction from the Markov chain (where the number of states is discrete) is quite evident. The role of the probability transition matrix in characterizing this chain is now played by the integral operator kernel ... 

Noteworthy that all the above formulated results can be applied to calculate the statistical characteristics of the products of polycondensation of an arbitrary mixture of monomers with kinetically independent groups under any regime of this process. To determine the values of the elements of the probability transition matrix of corresponding Markov chains it will suffice to calculate only the concentrations Q()- of chemical bonds (ij) at different conversions of functional groups. In the case of equilibrium polycondensation the concentrations Qy are controlled by the thermodynamic parameters, whereas under the nonequilibrium regime of this process they depend on kinetic parameters. 

Besides the fugacity models, the environmental science literature reports the use of models based on Markov chain principle to evaluate the environmental fate of chemicals in multimedia environment. Markov chain is a random process, and its theory lies in using transition matrix to describe the transition of a substance among different states [39,40]. If the substance has all together n different kinds of states,... 

Berthiaux H, Mizonov V, Zhukov V (2005) Application of the theory of Markov chains to model different processes in particle technology. Powder Technol 157 128-137... 

For many synthetic copolymers, it becomes possible to calculate all desired statistical characteristics of their primary structure, provided the sequence is described by a Markov chain. Although stochastic process 31 in the case of proteinlike copolymers is not a Markov chain, an exhaustive statistic description of their chemical structure can be performed by means of an auxiliary stochastic process 3iib whose states correspond to labeled monomeric units. As a label for unit M , it was suggested [23] to use its distance r from the center of the globule. The state of this stationary stochastic process 31 is a pair of numbers, (a, r), the first of which belongs to a discrete set while the second one corresponds to a continuous set. Stochastic process ib is remarkable for being stationary and Markovian. The probability of the transition from state a, r ) to state (/i, r") for the process of conventional movement along a heteropolymer macromolecule is described by the matrix-function of transition intensities... 

Random walks are often called Markov random walks. A Markov chain is a sequence of random events described in terms of a probability that the event under scmtiny evolved from a defined predecessor. In effect there is no memory of any preceding step in a Markov chain. Hidden Markov processes involve some short-term memory of preceding steps. 

Markov chains theory provides a powerful tool for modeling several important processes in electrochemistry and electrochemical engineering, including electrode kinetics, anodic deposit formation and deposit dissolution processes, electrolyzer and electrochemical reactors performance and even reliability of warning devices and repair of failed cells. The way this can be done using the elegant Markov chains theory is described in lucid manner by Professor Thomas Fahidy in a concise chapter which gives to the reader only the absolutely necessary mathematics and is rich in practical examples. 

Modeling Electrochemical Phenomena via Markov Chains and Processes... 

Markov Chain Evolution for the Anodic Deposit Formation<->Deposit Dissolution Process. The Initial State is Purely Ionic (/> ° = 1 p2 = 0)... 

Unidirectional kinetic processes cannot be immediately interpreted as Markov chains, since only the (1,1) element of the /- -matrix would differ from zero, violating the stochastic matrix constraints (Section II. 1). An artificial Markov matrix complying with this constraint can be visualized, however, with the understanding that no other element of this imbedded P-matrix, past the (1,1) element, will have a physical meaning. It follows that the initial state probability vector is non-zero only in its (1,1)... 

For Markov chains with two states 0,1, it turns out that we always have the equality = h so that they are not appropriate to model nonequilibrium processes. 

---