Gibbs Sampling

All that mattered was that slowly, by degrees, by left and right then left and right again, he was guiding them towards the destination.

Ian Rankin, Hide and Seek

There exist a number of algorithms that can give us a proposal density for our Markov chain’s dynamics. By using the Metropolis-Hastings algorithm we can obtain an acceptance probability for each proposed change of state within the Markov Chain. However, in some cases where we only know the conditional distribution between the variables, Gibbs sampling MCMC algorithm will be more efficient and helpful. 

Gibbs Illustration