Approximate Bayesian computation via regression density estimation

Abstract

Approximate Bayesian computation (ABC) methods, which are applicable when the likelihood is difficult or impossible to calculate, are an active topic of current research. Most current ABC algorithms directly approximate the posterior distribution, but an alternative, less common strategy is to approximate the likelihood function. This has several advantages. First, in some problems, it is easier to approximate the likelihood than to approximate the posterior. Second, an approximation to the likelihood allows reference analyses to be constructed based solely on the likelihood. Third, it is straightforward to perform sensitivity analyses for several different choices of prior once an approximation to the likelihood is constructed, which needs to be done only once. The contribution of the present paper is to consider regression density estimation techniques to approximate the likelihood in the ABC setting. Our likelihood approximations build on recently developed marginal adaptation density estimators by extending them for conditional density estimation. Our approach facilitates reference Bayesian inference, as well as frequentist inference. The method is demonstrated via a challenging problem of inference for stereological extremes, where we perform both frequentist and Bayesian inference. Copyright © 2013 John Wiley & Sons Ltd

Mining gold from implicit models to improve likelihood-free inference

“…We use a simple network architecture with a single hidden layer, 10 hidden units, and tanh activations. Figure 3 shows the mean squared error between logr(x|θ 0 , θ 1 ) and the true log r(x|θ 0 , θ 1 ) (estimated from histograms of 2·10 4 simulations from θ 0 =−0.8 and θ 1 =−0.6), summing over x ∈ [5,15], versus the training sample size (which refers to the total number of x samples, distributed over 10 values of θ ∈ [−1, −0.4]). We find that both Scandal and Rascal are dramatically more sample efficient than pure neural density estimation and the likelihood ratio trick, which do not leverage the joint score.…”

Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology

“…Density Estimation Likelihood-Free Inference works by learning a parameterized model for the joint density P(θ, d), from a set of samples drawn from that density (Bonassi et al 2011;Fan et al 2013;Papamakarios & Murray 2016). In its simplest form, we start by generating a set of samples {θ, d} from P(θ, d) by drawing parameters from the prior and forward simulating mock data:…”

“…With the two-step compression scheme defined, we then introduce Density Estimation Likelihood-Free Inference (delfi; Bonassi et al 2011;Fan et al 2013;Papamakarios & Murray 2016), which learns a parameterized model for the joint density of the parameters and compressed statistics P(θ, t), from which we can extract the posterior density by simply evaluating the joint density at the observed data t o , ie., P(θ |t o ) ∝ P(θ, t = t o ). We will show that high-fidelity posterior inference can be achieved with orders of magnitude fewer forward simulations than, for example, available implementations of Population Monte Carlo ABC (pmc-abc), making likelihood-free inference feasible for full-scale cosmological data analyses where simulations are expensive.…”

Likelihood-free approximate Gibbs sampling

“…See e.g. Fearnhead and Prangle (2012), Fan et al (2013) for other strategies. For each sample φ (i) = (f (i) , q (i) , n (i) ), i = 1, .…”

“…In contrast to Kousathanas et al (2016), synthetic datasets are not generated during each sampler iteration, thereby providing efficiencies for expensive simulator models, and only require sufficient synthetic datasets to adequately construct the full conditional models (e.g. Fan et al 2013). Construction of the approximate conditional distributions can exploit known structures of the high-dimensional posterior, where available, to considerably reduce computational overheads.…”