On the error in Laplace approximations of high-dimensional integrals

Ogden, Helen

doi:10.48550/arxiv.1808.06341

Cited by 2 publications

(9 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…. , w n ) is a set of nonnegative weights (this or a similar idea appear in Madigan et al, 2002;Feldman et al, 2011;Zhang et al, 2016;Huggins et al, 2016;Lucic et al, 2018;Campbell & Broderick, 2017, 2018. This log-likelihood approximation induces a coreset posterior approximation for Π given by…”

Section: Applicationsmentioning

confidence: 92%

“…The Fisher divergence has been used to prove central limit theorems (Johnson, 2004;Johnson & Barron, 2004) and as an objective for density estimation (Sriperumbudur et al, 2017;Hyvarinen, 2005). Special cases of the (p, ν)-Fisher distance have also been used in a Bayesian context both for analyzing approximation quality (Huggins & Zou, 2017; and as an objective function for approximate inference (Campbell & Broderick, 2017, 2018). We will discuss some of these applications in detail in Section 6.…”

Section: Wasserstein Distance Bounds Via the (P ν)-Fisher Normmentioning

confidence: 99%

“…These include the Laplace approximation (Schervish, 1995) and the integrated nested Laplace approximation (Rue et al, 2009(Rue et al, , 2017, approximate Bayesian computation (Marjoram et al, 2003;Marin et al, 2011;Karabatsos & Leisen, 2018), subsampling Markov chain Monte Carlo (Welling & Teh, 2011;Korattikara et al, 2014;Bardenet et al, 2014;Alquier et al, 2016a;Teh et al, 2016;Vollmer et al, 2016), consensus methods (Scott et al, 2013;Rabinovich et al, 2015;Srivastava et al, 2015;Li et al, 2017), and variational approaches (Blei et al, 2017) such as automatic differentiation variational inference (Ranganath et al, 2014;Kucukelbir et al, 2015). While these methods have empirically demonstrated computational gains on problems of interest, rigorous characterization of their finite-data approximation accuracy remains underdeveloped, though ongoing (Alquier et al, 2016b;Alquier & Ridgway, 2017;Ogden, 2017;Chérief-Abdellatif & Alquier, 2018;Wang & Blei, 2018;Ogden, 2018;Pati et al, 2018). We aim to provide theoretical tools to help address this gap.…”

Section: Introductionmentioning

confidence: 99%

“…Unfortunately, though, the Wasserstein distance is challenging to use in practice. To address this shortcoming, we introduce the (p, ν)-Fisher distance, which generalizes a number of existing distances in a range of literatures (Johnson, 2004;Johnson & Barron, 2004;Hyvarinen, 2005;Bolley et al, 2012;Sriperumbudur et al, 2017;Huggins & Zou, 2017;Campbell & Broderick, 2017, 2018. We extend and synthesize the results of Bolley et al (2012) and Huggins & Zou (2017) to show that the (p, ν)-Fisher distance provides an upper bound on the p-Wasserstein distance in many cases of interest.…”

Section: Introductionmentioning

confidence: 99%

“…We demonstrate the practicality of our proposed (p, ν)-Fisher distance by using it to analyze two scalable Bayesian approximation methods: the Laplace approximation (Schervish, 1995) and Bayesian coresets (Campbell & Broderick, 2017, 2018. First, we derive computable bounds on the p-Wasserstein distance between the exact posterior and the Laplace approximation for Bayesian models with log posterior densities that are strongly convex and have bounded third derivatives.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Practical bounds on the error of Bayesian posterior approximations: A nonasymptotic approach

Huggins,

Campbell,

Kasprzak

et al. 2018

Preprint

View full text Add to dashboard Cite

Bayesian inference typically requires the computation of an approximation to the posterior distribution. An important requirement for an approximate Bayesian inference algorithm is to output high-accuracy posterior mean and uncertainty estimates. Classical Monte Carlo methods, particularly Markov Chain Monte Carlo, remain the gold standard for approximate Bayesian inference because they have a robust finite-sample theory and reliable convergence diagnostics. However, alternative methods, which are more scalable or apply to problems where Markov Chain Monte Carlo cannot be used, lack the same finite-data approximation theory and tools for evaluating their accuracy. In this work, we develop a flexible new approach to bounding the error of mean and uncertainty estimates of scalable inference algorithms. Our strategy is to control the estimation errors in terms of Wasserstein distance, then bound the Wasserstein distance via a generalized notion of Fisher distance. Unlike computing the Wasserstein distance, which requires access to the normalized posterior distribution, the Fisher distance is tractable to compute because it requires access only to the gradient of the log posterior density. We demonstrate the usefulness of our Fisher distance approach by deriving bounds on the Wasserstein error of the Laplace approximation and Hilbert coresets. We anticipate that our approach will be applicable to many other approximate inference methods such as the integrated Laplace approximation, variational inference, and approximate Bayesian computation.

show abstract

Section: Applicationsmentioning

confidence: 92%