Likelihood-free approximate Gibbs sampling

Abstract

Likelihood-free methods such as approximate Bayesian computation (ABC) have extended the reach of statistical inference to problems with computationally intractable likelihoods. Such approaches perform well for small-to-moderate dimensional problems, but suffer a curse of dimensionality in the number of model parameters. We introduce a likelihood-free approximate Gibbs sampler that naturally circumvents the dimensionality issue by focusing on lower-dimensional conditional distributions. These distributions are estimated by flexible regression models either before the sampler is run, or adaptively during sampler implementation. As a result, and in comparison to Metropolis-Hastings based approaches, we are able to fit substantially more challenging statistical models than would otherwise be possible. We demonstrate the sampler's performance via two simulated examples, and a real analysis of Airbnb rental prices using a intractable high-dimensional multivariate non-linear state space model containing 13,140 parameters, which presents a real challenge to standard ABC techniques.

Benchmarking Simulation-Based Inference

“…We focused on approaches using neural networks for density estimation and did not compare to alternatives using Gaussian Processes (e.g., Meeds and Welling, 2014;Wilkinson, 2014). There are many other algorithms which the benchmark is currently lacking (e.g., Nott et al, 2014;Ong et al, 2018;Clarté et al, 2020;Prangle, 2019;Picchini et al, 2020;Radev et al, 2020;Rodrigues et al, 2020). Keeping our initial selection small allowed us to carefully investigate hyperparameter choices.…”

“…In recent years, several new SBI algorithms have been developed (e.g., Prangle, 2019;Järvenpää et al, 2020;Picchini et al, 2020;Rodrigues et al, 2020;, energized, in part, by advances in probabilistic machine learning (Rezende and Mohamed, 2015;. Despite-or possibly because-of these rapid and exciting developments, it is currently difficult to assess how different approaches relate to each other theoretically and empirically: First, different studies often use different tasks and metrics for comparison, and comprehensive comparisons on multiple tasks and simulation budgets are rare.…”

ABC of the Future

“…MCMC procedures can also used to generate samples from the approximate posterior distributions [Marjoram et al, 2003, Marjoram, 2013, Vihola and Franks, 2020, and improving their applicability to high-dimensional problems is an on-going research problem [Rodrigues et al, 2020, Clarté et al, 2020.…”

“…Although the rejection ABC algorithm is still being used frequently as a comparison method in the ABC and LFI literature, there are few applications where it would not be beneficial to instead take advantage of more sophisticated versions of this basic algorithm [Beaumont et al, 2009, Blum, 2010, Csilléry et al, 2010, Marin et al, 2012, Moral et al, 2012, Clarté et al, 2020, Rodrigues et al, 2020. The ABC-PMC approach by Beaumont et al [2009] is an extension of the rejection ABC algorithm based on importance sampling, and aims to improve the efficiency of the procedure by retrieving a sequence of intermediate distributions.…”

Componentwise approximate Bayesian computation via Gibbs-like steps

“…Kousathanas et al (2016) also run a Gibbs-like ABC algorithm that assumes the availability of conditionally sufficient statistics to preserve the coherence of the algorithm. Rodrigues et al (2020) propose another Gibbs-like ABC algorithm in which the conditional distributions are approximated by regression models.…”