Unbiased MLMC-based variational Bayes for likelihood-free inference

Abstract

Variational Bayes (VB) is a popular tool for Bayesian inference in statistical modeling. Recently, some VB algorithms are proposed to handle intractable likelihoods with applications such as approximate Bayesian computation. In this paper, we propose several unbiased estimators based on multilevel Monte Carlo (MLMC) for the gradient of Kullback-Leibler divergence between the posterior distribution and the variational distribution when the likelihood is intractable, but can be estimated unbiasedly. The new VB algorithm differs from the VB algorithms in the literature which usually render biased gradient estimators. Moreover, we incorporate randomized quasi-Monte Carlo (RQMC) sampling within the MLMC-based gradient estimators, which was known to provide a favorable rate of convergence in numerical integration. Theoretical guarantees for RQMC are provided in this new setting. Numerical experiments show that using RQMC in MLMC greatly speeds up the VB algorithm, and finds a better parameter value than some existing competitors do.

An adaptive mixture-population Monte Carlo method for likelihood-free inference

“…We consider the logistic regression with Six City data set [26], which was studied in [11] and [12] in the context of variational Bayes. The data consist of binary responses y ij which is the wheezing status (1 if wheezing, 0 if not wheezing) of the ith child at time-point j, where i = 1, .…”

“…This differs from the usual way of minimizing KL(q(θ) π(θ)). To recover VI based on KL(q(θ) π(θ)) in the likelihood-free setting, [12] uses multilevel Monte Carlo [13,14] to derive unbiased estimates of the gradient of the KL. On the other hand, [15] modified the VBIL method to work with unbiased log-likelihood estimates in the synthetic likelihood framework, resulting in the variational Bayes synthetic likelihood (VBSL) method.…”