Markov chain theory

Markov chains describe a sequence Xofn observations of a random variable Z (Dove-ton, 1994 Tuckwell, 1995)  

This is formalised by the one-step transition probability pr j of adjacent events (Clarke and Disney, 1970)  

The one-step transition probabilities can be obtained from the empirical transition frequencies. All possible transitions from one state to another are summarised in the transition probability matrix P  

Row 1 denotes the probabilities of a one step transition from state 1 to any other state 1. .k. The sum of row entries equals unity. The total probability Pr a of a specific sequence X is given as the product of the involved one-step transition probabilities  

---

Markov chains theory provides a powerful tool for modeling several important processes in electrochemistry and electrochemical engineering, including electrode kinetics, anodic deposit formation and deposit dissolution processes, electrolyzer and electrochemical reactors performance and even reliability of warning devices and repair of failed cells. The way this can be done using the elegant Markov chains theory is described in lucid manner by Professor Thomas Fahidy in a concise chapter which gives to the reader only the absolutely necessary mathematics and is rich in practical examples. 

See, e.g., J.G. Kemeny and J.L. Snell, Finite Markov Chains (Van Nostrand, Princeton 1960) D.L. Isaacson and R.W. Madsen, Markov Chains, Theory and Applications (Wiley, New York 1976). 

When the Markov character of unit sequence distribution in the copolymer is established and the elements of matrix Q are known, the standard procedure of Markov chain theory allows one to obtain the explicit formulae for all the statistical characteristics of the copolymer fraction obtained at given monomer feed composition Xj by means of the simple algebraic operations [51-53, 6J. 

In reality, such a simulation is not necessary even for the determination of the validity limits of the above statistical approach to the quantitative description of the copolymers which is based on the formalism of the stationary Markov chain theory. 

The Markovian character of the sequence distribution statistics in the macromolecules results [6, 94] from assumption about the steady-state of the radical concentrations, which usually holds with a high degree of accuracy in the copolymerization processes [6, 95], It is worth mentioning that along with such kinetic stationarity one should usually speak about the statistical stationarity. It means that when the number of the units in copolymer molecules exceeds 10-15, their composition practically becomes independent on degree of polymerization and is indistinguishable from the value predicted by the stationary Markov chain theory. This conclusion is supported by the theoretical [96,97,6] and experimental [98] evidence. 

The parameters a = l/rij5 the number of which equals m(m — IX are reciprocal reactivity ratios (2.8) of binary copolymers. Markov chain theory allows one, without any trouble, to calculate at any m, all the necessary statistical characteristics of the copolymers, which are formed at given composition x of the monomer feed mixture. For instance, the instantaneous composition of the multicomponent copolymer is still determined by means of formulae (3.7) and (3.8), the sums which now contain m items. In the general case the problems of the calculation of the instantaneous values of sequence distribution and composition distribution of the Markov multicomponent copolymers were also solved [53, 6]. The availability of the simple algebraic expressions puts in question the expediency of the application of the Monte-Carlo method, which was used in the case of terpolymerization [85,99-103], for the calculations of the above statistical characteristics. Actually, the probability of any sequence MjMjWk. .. Mrl 4s of consecutive monomer units, selected randomly from a polymer chain is calculated by means of the elementary formula ... 

It is worth noting that the above-mentioned expressions (4.5-4.7) contain, as particular cases, the results obtained both for binary [54] and ternary [112] copolymerization. However, the general formulae (4.6) and (4.7) for indexes of sequential homogeneity of multicomponent copolymers with any m were not obtained earlier by the author of Refs. [Ill, 113], who investigated this problem theoretically. The approaches applied in the above papers result in cumbersome formulae and are not needed since Eqs. (4.6) and (4.7) can be immediately obtained [6] from the Markov chain theory. 

The process described above is thus repeated with constant time intervals. So, we have a discrete time t = nAr where n is the number of displacement steps. By the rules of probability balance and by the prescriptions of the Markov chain theory, the probability that shows a particle in position i after n motion steps and having a k-type motion is written as follows ... 

The basic elements of Markov-chain theory are the state space, the one-step transition probability matrix or the policy-making matrix and the initial state vector termed also the initial probability function In order to develop in the following a portion of the theory of Markov chains, some definitions are made and basic probability concepts are mentioned. 

Recent applications of the Markov chain theory in geology have introduced continuous-lag modelling of spatial variability (Carle and Fogg, 1997 Fogg et al., 1998). The continuous-lag approach extends the probability of state transitions recorded at fixed intervals (discrete-lag) to any desired interval by considering conditional rates of... 

MARKOV CHAIN THEORY DEFINITION OF THE PROBABILITY MATRIX... 

Markov Chain Theory Definition of the ProhabiUty Matrix 239... 

From the finite Markov chain theory, it follows that the atomic fiactions x and x ) of atoms A and B in the alloy can be calculated with the probabilities and the following equations... 

To apply the Markov chain theory to experimental examples, experimental data must be available between the composition of an alloy and the concentrations of the components in the electrolyte. Examples can be found in a book by Watanabe. Some representative systems will be described. 

The end of the chain growths is achieved if another inert molecule terminates the process. The Markov chain theory can be applied on this reaction scheme (Chapter 7). The reactivity of the radical cations A+ and B+ with a polymer chain are described by rate equations ... 

The propositions (4.1) and (4.2) provide the basis for computation of p(°)[Z2 ( )2 = l],z = (0,0,0), compare the semi-formal (1). The computation is a standard procedure in Markov Chain theory only multiplication of the associated transition matrices is required. For an example see figure 4. 

Statistical methods which have been used to treat polymerization problems can be grouped into three categories direct, formal Markov chain theory, and recursive approaches. They differ in detail and objectives but all contain the underlying assumption that polymerization follows Markovian statistics. 

Example 3.3 Find the number-average degree of polymerization and the PCLD resulting from an AA + BB step-growth polymerization, using the Markov chain theory. 

Bilsel (2009) developed a risk detectability model based on Markov Chain theory. His work was motivated by Petri-net based model to analyze propagation of disruption information (Wu et al., 2007). We will present the Markov Chain model of Bilsel and Ravindran (2012) next. 

Calculations. The statistical analysis of the copolymer sequencing was performed in terms of Markov chain theory, using the 2nd order Markov chain model. The equations were numerically solved with respect to the conditional probabilities by least squares minimization of the deviations of calculated and measured pentad contributions. 

---