5 Continuous Markov chains 5.3 Asymptotic and invariant distributions 5.5 Use of the rate matrix

5.4 Detailed balance

Analalogously to discrete-time Markov chains a system satisfies detailed balance if there is a distribution on the states such that the probability of being in state $i$ and moving (in infinitesimal time $h$ ) to state $j$ is equal to the probability of being in state $j$ and moving (in time $h$ ) to state $i$ .

\pi_{i}Q_{ij}h=\pi_{j}Q_{ji}h~{}~{}~{}\mbox{or simply}~{}~{}~{}\pi_{i}Q_{ij}=% \pi_{j}Q_{ji}.

Theorem 5.4.1.

If a continuous Markov chain satisfies detailed balance as above then $\pi$ is its stationary distribution.

Proof.

See tutorial worksheet.

Suppose $\pi$ satisfies detailed balance. Then

[\pi Q]_{j}=\sum_{i=1}^{m}\pi_{i}Q_{ij}=\sum_{i=1}^{m}\pi_{j}Q_{ji}=\pi_{j}% \sum_{i=1}^{m}Q_{ji}=0.

∎

As with discrete time Markov chains the interpretation of detailed balance is that at equilibrium it would be impossible to tell whether the system was going forward or backwards in time.

Theorem 5.4.2.

Consider a continuous Markov chain on statespace $\{0,1,2,\dots,N\}$ with rate matrix $Q$ . If the only non-zero elements of $Q$ are exactly those immediately above, on, and immediately below the main diagonal then the Markov chain satisfies detailed balance.

Proof.

We will find a distribution $\pi$ which satisfies all of the detailed balance equations.

Q=\left(\begin{array}[]{cccccccc}-Q_{01}&Q_{01}&0&0&0&\dots&0&0\\ Q_{10}&-(Q_{10}+Q_{12})&Q_{12}&0&0&\dots&0&0\\ 0&Q_{21}&-(Q_{21}+Q_{23})&Q_{23}&0&\dots&0&0\\ 0&0&Q_{32}&-(Q_{32}+Q_{34})&Q_{34}&\dots&0&0\\ \vdots\\ 0&0&0&0&0&\dots&Q_{N,N-1}&-Q_{N,N-1}\end{array}\right).

Since the only non-zero terms are on the super- and sub-diagonal, detailed balance requires exactly that

\pi_{i-1}Q_{i-1,i}=\pi_{i}Q_{i,i-1},~{}~{}~{}(i=1,\dots,N).

Or equivalently, since the sub-diagonal terms are non-zero,

\pi_{i}=\frac{Q_{i-1,i}}{Q_{i,i-1}}\pi_{i-1}~{}~{}~{}i=1,\dots,N.

Therefore

\pi_{i}=\pi_{0}\prod_{j=1}^{i}\frac{Q_{j-1,j}}{Q_{j,j-1}}~{}~{}~{}i=1,\dots,N.

Since the probabilities must sum to one:

1=\sum_{k=0}^{N}\pi_{k}=\pi_{0}\left(1+\sum_{k=1}^{N}\prod_{j=1}^{k}\frac{Q_{j% -1,j}}{Q_{j,j-1}}\right).

Therefore

\pi_{0}=\left(1+\sum_{k=1}^{N}\prod_{j=1}^{k}\frac{Q_{j-1,j}}{Q_{j,j-1}}\right% )^{-1}.

and

\pi_{i}=\frac{\prod_{j=1}^{i}\frac{Q_{j-1,j}}{Q_{j,j-1}}}{1+\sum_{k=1}^{N}% \prod_{j=1}^{k}\frac{Q_{j-1,j}}{Q_{j,j-1}}},~{}~{}~{}(i=1,\dots,N).

∎

Remark.

The above proof in fact only requires that the sub-diagonal (and therefore the diagonal) entries be non-zero. If any of the super-diagonal elements, $Q_{i-1,i}$ , are zero then all states from $i$ upwards are transient and have $\pi_{i}=0$ . It is relatively straightforward to generalise the proof to allow any of the elements of $Q$ to be zero (subject to $\sum_{j}Q_{ij}=0$ ) by considering the smaller reducible chains that this creates.