Accelerating Bayesian Synthetic Likelihood With the Graphical Lasso

An, Ziwen; South, Leah F.; Nott, David J.; Drovandi, Christopher C.

doi:10.1080/10618600.2018.1537928

Cited by 23 publications

(25 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A key feature of the proposed method is the automated selection and combination of summary statistics from a large pool of candidates. While there are several works on summary statistics selection in the framework of approximate Bayesian computation (Aeschbacher et al, 2012;Fearnhead and Prangle, 2012;Blum et al, 2013;Gutmann et al, 2014;Marin et al, 2016;Gutmann et al, 2018;Jiang et al, 2018), there is comparably little corresponding work on synthetic likelihood (Wood, 2010) with the exception of the recent work by An et al (2019) and Ong et al (2018) whose robust estimation techniques of the (inverse) covariance matrix are broadly related to summary statistics selection. We have shown that synthetic likelihood is a special case of the proposed approach so that our techniques for summary statistics selection could also be used there.…”

Section: Discussionmentioning

confidence: 99%

“…The cross-validation adds computational cost and the dependency of λ min on θ can make more detailed theoretical investigations more difficult. In order to reduce the cost or to facilitate theoretical analyses, working with a fixed value of λ as, for example, An et al (2019) for synthetic likelihood with the graphical lasso may be appropriate.…”

Section: Data-driven Selection Of Summary Statisticsmentioning

confidence: 99%

“…Linear LFIRE has the additional cost of constructing the set X m once, and the cost of performing penalised logistic regression for each θ, including potentially computationally intensive cross-validation to establish the regularisation strength. Synthetic likelihood, on the other hand, requires inversion of the covariance matrix of the summary statistics for each θ. Cross-validation can also be used for regularisation of the synthetic likelihood with a graphical lasso for robust inference in situations with a large or poorly-conditioned covariance matrix (An et al, 2019). When the covariance matrix is well-conditioned, penalised logistic regression with cross-validation is more expensive than standard inversion of the covariance matrix, and the difference in computational cost can be seen as the price that needs to be paid for the relaxation of the Gaussianity assumption, and for feature selection through regularised inference.…”

Section: Computational Aspectsmentioning

confidence: 99%

See 2 more Smart Citations

Likelihood-Free Inference by Ratio Estimation

et al. 2022

View full text Add to dashboard Cite

We consider the problem of parametric statistical inference when likelihood computations are prohibitively expensive but sampling from the model is possible. Several so-called likelihood-free methods have been developed to perform inference in the absence of a likelihood function. The popular synthetic likelihood approach infers the parameters by modelling summary statistics of the data by a Gaussian probability distribution. In another popular approach called approximate Bayesian computation, the inference is performed by identifying parameter values for which the summary statistics of the simulated data are close to those of the observed data. Synthetic likelihood is easier to use as no measure of "closeness" is required but the Gaussianity assumption is often limiting. Moreover, both approaches require judiciously chosen summary statistics. We here present an alternative inference approach that is as easy to use as synthetic likelihood but not as restricted in its assumptions, and that, in a natural way, enables automatic selection of relevant summary statistic from a large set of candidates. The basic idea is to frame the problem of estimating the posterior as a problem of estimating the ratio between the data generating distribution and the marginal distribution. This problem can be solved by logistic regression, and including regularising penalty terms enables automatic selection of the summary statistics relevant to the inference task. We illustrate the general theory on canonical examples and employ it to perform inference for challenging stochastic nonlinear dynamical systems and high-dimensional summary statistics.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Data-driven Selection Of Summary Statisticsmentioning

confidence: 99%

Section: Computational Aspectsmentioning

confidence: 99%

See 1 more Smart Citation

Likelihood-Free Inference by Ratio Estimation

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Despite the relative advantages and efficiencies of BSL, and recent work in this area (e.g. Nott et al, 2019;An et al, 2019b;Ong et al, 2018b) there remain some key inefficiencies in the method. Most prominently, for a Gaussian synthetic likelihood the unknown mean and covariance matrix must be estimated by simulation for every proposed parameter within any inference algorithm.…”

Section: Introductionmentioning

confidence: 99%

“…A number of efficient covariance matrix estimation techniques have been considered to reduce the needed number of model simulations in BSL. An et al (2019b) use the graphical lasso to provide a sparse estimate of the precision matrix. However, performance is inhibited when there is a low degree of sparsity in the covariance or inverse covariance matrix.…”

Section: Introductionmentioning

confidence: 99%

Efficient Bayesian synthetic likelihood with whitening transformations

Priddle¹,

Sisson²,

Frazier³

et al. 2019

Preprint

View full text Add to dashboard Cite

Likelihood-free methods are an established approach for performing approximate Bayesian inference for models with intractable likelihood functions. However, they can be computationally demanding. Bayesian synthetic likelihood (BSL) is a popular such method that approximates the likelihood function of the summary statistic with a known, tractable distribution -typically Gaussian -and then performs statistical inference using standard likelihood-based techniques. However, as the number of summary statistics grows, the number of model simulations required to accurately estimate the covariance matrix for this likelihood rapidly increases. This poses significant challenge for the application of BSL, especially in cases where model simulation is expensive. In this article we propose whitening BSL (wBSL) -an efficient BSL method that uses approximate whitening transformations to decorrelate the summary statistics at each algorithm iteration. We show empirically that this can reduce the number of model simulations required to implement BSL by more than an order of magnitude, without much loss of accuracy. We explore a range of whitening procedures and demonstrate the performance of wBSL on a range of simulated and real modelling scenarios from ecology and biology.

show abstract

Bayesian Synthetic Likelihood

Frazier,

Drovandi,

Nott

2023

Wiley StatsRef: Statistics Reference Online

View full text Add to dashboard Cite

Bayesian statistics is concerned with conducting posterior inference for the unknown quantities in a given statistical model. Conventional Bayesian inference requires the specification of a probabilistic model for the observed data, and the construction of the resulting likelihood function. However, sometimes the model is so complicated that evaluation of the likelihood is infeasible, which renders exact Bayesian inference impossible. Bayesian synthetic likelihood (BSL) is a posterior approximation procedure that can be used to conduct inference in situations where the likelihood is intractable, but where simulation from the model is straightforward. In this article, we give a high‐level presentation of BSL, and its extensions aimed at delivering scalable and robust posterior inferences.

show abstract

Accelerating Bayesian Synthetic Likelihood With the Graphical Lasso

Cited by 23 publications

References 12 publications

Likelihood-Free Inference by Ratio Estimation

Likelihood-Free Inference by Ratio Estimation

Efficient Bayesian synthetic likelihood with whitening transformations

Bayesian Synthetic Likelihood

Contact Info

Product

Resources

About