Parallel Gaussian Process Surrogate Bayesian Inference with Noisy Likelihood Evaluations

Järvenpää, Marko; Gutmann, Michael U.; Vehtari, Aki; Marttinen, Pekka

doi:10.1214/20-ba1200

Cited by 26 publications

(58 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since standard ABC is notoriously slow and requires tuning of some hyperparameters, there has been considerable research in making likelihood-free inference more efficient, using, for example, ideas from Bayesian optimisation and experimental design (Gutmann and Corander, 2016;Järvenpää et al, 2019Järvenpää et al, , 2020, conditional density estimation (Papamakarios and Murray, 2016;Lueckmann et al, 2017;Greenberg et al, 2019), classification (Gutmann et al, 2018), indirect inference (Drovandi et al, 2015), optimisation (Meeds and Welling, 2015;Ikonomov and Gutmann, 2020), and more broadly surrogate modelling with Gaussian processes (Wilkinson, 2014;Meeds and Welling, 2015) and neural networks (Blum and Francois, 2010;Chen and Gutmann, 2019;Papamakarios et al, 2019).…”

Section: Likelihood-free Inferencementioning

confidence: 99%

“…Likelihood-free inference has gained much traction recently, with many methods leveraging advances in machinelearning (e.g. Gutmann and Corander, 2016;Lueckmann et al, 2017;Järvenpää et al, 2018;Chen and Gutmann, 2019;Papamakarios et al, 2019;Thomas et al, 2020;Järvenpää et al, 2021). Ultimately, however, the quality of the statistical inference within a scientific downstream task depends on the data that are available in the first place.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sequential Bayesian Experimental Design for Implicit Models via Mutual Information

2021

Self Cite

View full text Add to dashboard Cite

Bayesian experimental design (BED) is a framework that uses statistical models and decision making under uncertainty to optimise the cost and performance of a scientific experiment. Sequential BED, as opposed to static BED, considers the scenario where we can sequentially update our beliefs about the model parameters through data gathered in the experiment. A class of models of particular interest for the natural and medical sciences are implicit models, where the data generating distribution is intractable, but sampling from it is possible. Even though there has been a lot of work on static BED for implicit models in the past few years, the notoriously difficult problem of sequential BED for implicit models has barely been touched upon. We address this gap in the literature by devising a novel sequential design framework for parameter estimation that uses the Mutual Information (MI) between model parameters and simulated data as a utility function to find optimal experimental designs, which has not been done before for implicit models. Our approach uses likelihood-free inference by ratio estimation to simultaneously estimate posterior distributions and the MI. During the sequential BED procedure we utilise Bayesian optimisation to help us optimise the MI utility. We find that our framework is efficient for the various implicit models tested, yielding accurate parameter estimates after only a few iterations.

show abstract

Section: Likelihood-free Inferencementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Sequential Bayesian Experimental Design for Implicit Models via Mutual Information

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Besides deep learning based approaches, another major machine learning inspired branch of the ABC literature, concerns log-likelihood and posterior approximations via Gaussian Process Surrogates (GPSs) (Meeds and Welling, 2014;Järvenpää et al, 2018Järvenpää et al, , 2021Acerbi, 2020). A major benefit of GPSs lies in the ability for clever training data selection via active learning, since such GPSs allow uncertainty quantification out of the box, which in turn can be utilized for the purpose of targeting high uncertainty regions in parameter space.…”

mentioning

confidence: 99%

Likelihood approximation networks (LANs) for fast inference of simulation models in cognitive neuroscience

Fengler

Govindarajan

Chen

et al. 2021

eLife

115

View full text Add to dashboard Cite

In cognitive neuroscience, computational modeling can formally adjudicate between theories and affords quantitative fits to behavioral/brain data. Pragmatically, however, the space of plausible generative models considered is dramatically limited by the set of models with known likelihood functions. For many models, the lack of a closed-form likelihood typically impedes Bayesian inference methods. As a result, standard models are evaluated for convenience, even when other models might be superior. Likelihood-free methods exist but are limited by their computational cost or their restriction to particular inference scenarios. Here, we propose neural networks that learn approximate likelihoods for arbitrary generative models, allowing fast posterior sampling with only a one-off cost for model simulations that is amortized for future inference. We show that these methods can accurately recover posterior parameter distributions for a variety of neurocognitive process models. We provide code allowing users to deploy these methods for arbitrary hierarchical model instantiations without further training.

show abstract

“…Illustrations of BOLFI are in Sections 6.1 and 6.2. A more recent contribution, exploiting GPs to predict a log-SL, is in Järvenpää et al (2020).…”

Section: Algorithmic Initialization Using Bolfi and Elfimentioning

confidence: 99%

Sequentially Guided MCMC Proposals for Synthetic Likelihoods and Correlated Synthetic Likelihoods

2023

View full text Add to dashboard Cite

Synthetic likelihood (SL) is a strategy for parameter inference when the likelihood function is analytically or computationally intractable. In SL, the likelihood function of the data is replaced by a multivariate Gaussian density over summary statistics of the data. SL requires simulation of many replicate datasets at every parameter value considered by a sampling algorithm, such as Markov chain Monte Carlo (MCMC), making the method computationally-intensive. We propose two strategies to alleviate the computational burden. First, we introduce an algorithm producing a proposal distribution that is sequentially tuned and made conditional to data, thus it rapidly guides the proposed parameters towards high posterior density regions. In our experiments, a small number of iterations of our algorithm is enough to rapidly locate high density regions, which we use to initialize one or several chains that make use of off-the-shelf adaptive MCMC methods. Our "guided" approach can also be potentially used with MCMC samplers for approximate Bayesian computation (ABC). Second, we exploit strategies borrowed from the correlated pseudo-marginal MCMC literature, to improve the chains mixing in a SL framework. Moreover, our methods enable inference for challenging case studies, when the posterior is multimodal and when the chain is initialised in low posterior probability regions of the parameter space, where standard samplers failed. To illustrate the advantages stemming from our framework we consider five benchmark examples, including estimation of parameters for a cosmological model and a stochastic model with highly non-Gaussian summary statistics.

show abstract

Parallel Gaussian Process Surrogate Bayesian Inference with Noisy Likelihood Evaluations

Cited by 26 publications

References 43 publications

Sequential Bayesian Experimental Design for Implicit Models via Mutual Information

Sequential Bayesian Experimental Design for Implicit Models via Mutual Information

Likelihood approximation networks (LANs) for fast inference of simulation models in cognitive neuroscience

Sequentially Guided MCMC Proposals for Synthetic Likelihoods and Correlated Synthetic Likelihoods

Contact Info

Product

Resources

About