Markov chains time-reversible

O. Cappe, C. P. Robert, and T. Ryden, Reversible jump, birth-and-death and more general continuous time Markov chain Monte Carlo samplers. JRSS Ser B Stat Method 65 679-700 (2003). 

In Section 5.1 we introduce the stochastic processes. In Section 5.2 we will introduce Markov chains and define some terms associated with them. In Section 5.3 we find the n-step transition probability matrix in terms of one-step transition probability matrix for time invariant Markov chains with a finite state space. Then we investigate when a Markov ehain has a long-run distribution and discover the relationship between the long-run distribution of the Markov chain and the steady state equation. In Section 5.4 we classify the states of a Markov chain with a discrete state space, and find that all states in an irreducible Markov chain are of the same type. In Section 5.5 we investigate sampling from a Markov chain. In Section 5.6 we look at time-reversible Markov chains and discover the detailed balance conditions, which are needed to find a Markov chain with a given steady state distribution. In Section 5.7 we look at Markov chains with a continuous state space to determine the features analogous to those for discrete space Markov chains. 

TIME-REVERSIBLE MARKOV CHAINS AND DETAILED BALANCE 117... 

Tlme-iW0rsible Msrkw chains. If we look at the states of a Markov chain in the reverse time order they also form a Markov chain called the backwards chain. Let the transition probabilities for the backwards chain be... 

The Markov chain is said to be time reversible when the backwards Markov chain and the forward Markov chain have the same transition probabilities. In other words q J = p j for all states i and j. Then it follows that the transition probabilities satisfy... 

Time reversible Markov chains satisfy the detailed balance condition. Let (x) be the steady state density and /(v x) be the density function of the one-step transitions... 

A Markov chain is time reversible if the one-step transition probabilities for the backwards chain where the states are taken in reverse time order are the same as the one-step probabilities of the (forwards) Markov chain. 

A time-reversible Markov chain has transition probabilities satisfying the detailed balance condition. 

These Metropolis-accepted or -rejected moves are taken in the distant past or far future in imaginary time, and ensure that the Markov chain converges to its target distribution. The algorithm s failure to meet the assumed criterion of microscopic reversibility causes the time-step bias to accumulate at the middle of the reptile, where the desired pure distribution is being sampled. Therefore, RQMC variants were introduced to avoid these difficulties [20-24]. 

---