Control Functionals for Monte Carlo Integration

Oates, Chris J.; Girolami, Mark; Chopin, Nicolás

doi:10.1111/rssb.12185

Cited by 138 publications

(236 citation statements)

References 72 publications

Supporting

Mentioning

234

Contrasting

Unclassified

Order By: Relevance

“…Many other control variates are used in practice [140], [146], [203]. We present another popular type of a control variate, the score function control variate, in Section 4.2.…”

Section: Control Variatesmentioning

confidence: 99%

Advances in Variational Inference

Zhang

Bütepage

Kjellström

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

484

343

View full text Add to dashboard Cite

Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem. This approach has been successfully applied to various models and large-scale applications. In this review, we give an overview of recent trends in variational inference. We first introduce standard mean field variational inference, then review recent advances focusing on the following aspects: (a) scalable VI, which includes stochastic approximations, (b) generic VI, which extends the applicability of VI to a large class of otherwise intractable models, such as non-conjugate models, (c) accurate VI, which includes variational models beyond the mean field approximation or with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks. Finally, we provide a summary of promising future research directions.

show abstract

“…Many other control variates are used in practice [140], [146], [203]. We present another popular type of a control variate, the score function control variate, in Section 4.2.…”

Section: Control Variatesmentioning

confidence: 99%

Advances in Variational Inference

Zhang

Bütepage

Kjellström

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

484

343

View full text Add to dashboard Cite

show abstract

“…As discused in Section 2.4, notable examples include Quasi Monte Carlo methods [26,42,17,18], Bayesian quadrature [48,9], and Kernel herding [11,5,10]. These methods have been studied extensively in recent years [55,8,45,46,4,62,30] and have recently found applications in, for instance, machine learning and statistics [3,32,21,9,31,43,50].…”

Section: Kernel-based Quadrature Rulesmentioning

confidence: 99%

“…Similar examples can be seen in applications to statistics and machine learning, as mentioned below. In these situations, one can only use a limited number of design points, and thus it is desirable to have quadrature rules with a faster convergence rate, in order to obtain a reliable solution [46].…”

Section: Introductionmentioning

confidence: 99%

Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings

2019

View full text Add to dashboard Cite

This paper presents convergence analysis of kernel-based quadrature rules in misspecified settings, focusing on deterministic quadrature in Sobolev spaces. In particular, we deal with misspecified settings where a test integrand is less smooth than a Sobolev RKHS based on which a quadrature rule is constructed. We provide convergence guarantees based on two different assumptions on a quadrature rule: one on quadrature weights, and the other on design points. More precisely, we show that convergence rates can be derived (i) if the sum of absolute weights remains constant (or does not increase quickly), or (ii) if the minimum distance between design points does not decrease very quickly. As a consequence of the latter result, we derive a rate of convergence for Bayesian quadrature in misspecified settings. We reveal a condition on design points to make Bayesian quadrature robust to misspecification, and show that, under this condition, it may adaptively achieve the optimal rate of convergence in the Sobolev space of a lesser order (i.e., of the unknown smoothness of a test integrand), under a slightly stronger regularity condition on the integrand. MSC 2010 subject classification: Primary: 65D30, Secondary: 65D32, 65D05, 46E35, 46E22.

show abstract

“…Various strategies have been proposed to reduce the variability in the Monte Carlo estimate of the expectation

E [f (ϑ)]

of a scalar function f of parameter ϑ with respect to some posterior distribution π ( ϑ ), including Rao–Blackwellization (Robert and Casella, ) and the control variates (Dellaportas and Kontoyiannis, ; Oates et al ., ). These techniques produce an efficient estimator of

E [f (ϑ)]

based on sampled ϑ generated from an MCMC sampler.…”

Section: A Deterministic Proposal Distributionmentioning

confidence: 99%

Efficient Data Augmentation for Multivariate Probit Models with Panel Data: An Application to General Practitioner Decision Making about Contraceptives

Chin

Gunawan

Fiebig

et al. 2020

Journal of the Royal Statistical Society Series C: Applied Statistics

View full text Add to dashboard Cite

Summary The paper considers the problem of estimating a multivariate probit model in a panel data setting with emphasis on sampling a high dimensional correlation matrix and improving the overall efficiency of the data augmentation approach. We reparameterize the correlation matrix in a principled way and then carry out efficient Bayesian inference by using Hamiltonian Monte Carlo sampling. We also propose a novel antithetic variable method to generate samples from the posterior distribution of the random effects and regression coefficients, resulting in significant gains in efficiency. We apply the methodology by analysing stated preference data obtained from Australian general practitioners evaluating alternative contraceptive products. Our analysis suggests that the joint probability of discussing combinations of contraceptive products with a patient shows medical practice variation among the general practitioners, which indicates some resistance even to discuss these products, let alone to recommend them.

show abstract

Control Functionals for Monte Carlo Integration

Cited by 138 publications

References 72 publications

Advances in Variational Inference

Advances in Variational Inference

Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings

Efficient Data Augmentation for Multivariate Probit Models with Panel Data: An Application to General Practitioner Decision Making about Contraceptives

Contact Info

Product

Resources

About