Irreducibility of Finite Markov Chains

The concept of ergodicity, and the method of analyzing it, can be understood using an analogy to a simpler finite model, a discrete Markov Chain. 

A discrete Markov Chain is a stochastic process in which, at each step, a transition is made from the current state i to another j selected from a given probability distribution. In the case of a finite collection of states S = 1,2. fc, we assume that the transition probabilities ny e [0,1] are given and denote the state at time n by S . Thus we have 

A distribution on the state space is typically associated to a row vector f = (V i, V 2, with nonzero elements and which is normalized such that 

This might be viewed as defining the relative proportion, in a large collection of walkers (independent realizations of a stochastic evolution), that are found in each state. As the walkers make moves based on the transition probabilities, the distribution will vary. We may denote hy =, V ,2, , the distribution 

This iteration can be viewed as analogous to dynamics. We may think of the evolution of the Markov chain as being realized through multiplication by the Markov matrix 77. Let jt represent the transition probability of being in state i at iteration I given that we were in state j at time 0, i.e. jt is the (ij) entry of the matrix IlK 

---