A recent Arxiv'd paper by Alexander Buchholz, Daniel Ahfock, and Sylvia Richardson (Distributed Computation for Marginal Likelihood based Model Choice) proposes a novel computational approach for Bayesian model choice in Big Data applications. Their approach is in line with recent research investigating the use of parallel computing power in Markov chain Monte Carlo (MCMC) methods (e.g. consensus Monte Carlo).

This challenging analysis is typically intractable for standard computational approaches because of the number of observations rather than the number of features in the dataset. By partitioning the data and running methods in parallel the computational workload is reduced. However, there remains a challenge in combining these parallel computations so that the resulting analysis is valid.

The model evidence, also known as the posterior normalising constant, is computationally challenging to compute but crucial quantity for Bayesian model choice algorithms. The key idea of this paper is to make use of an unbiased approximation of the model evidence of the full data from estimates of the model evidence on subsets of the data. This approximation is then used within standard MCMC methods such as a Gibbs sampling or reversible jump samplers. The Gibbs approach from the paper relies on conjugate distributions, while the reversible jump sampler is more general but relies on a Normal approximation of the posterior for each split. For this method, the authors provide an upper bound on the approximation error.

The methods proposed are demonstrated on several applications where the parallel computing architecture allows for a speedup of several orders of magnitude while incurring negligible bias. This includes applications to a real genetic dataset from the UK Biobank database with over 130,000 observations. Full details of the method including discussion of extensions and practical implementation advice are available in the paper.