Semi-exact control functionals from Sard’s method

South, Leah F.; Karvonen, Toni; Nemeth, Christopher; Oates, Chris J.

doi:10.1093/biomet/asab036

Cited by 10 publications

(12 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is on these types of CVs that vector-valued CVs are built [Sun et al, 2021]. Alternatively, one may take U to be a (parametric) set of neural networks [Wan et al, 2019, Si et al, 2021, or even a combination of neural networks and the aforementioned spaces [South et al, 2022a, Si et al, 2021. In this paper, we will focus on CVs constructed with neural networks, which are known as neural control variates (Neural-CVs).…”

Section: Control Variate Methodsmentioning

confidence: 99%

Meta-learning Control Variates: Variance Reduction with Limited Data

Sun¹,

Oates²,

Briol³

2023

Preprint

View full text Add to dashboard Cite

Control variates can be a powerful tool to reduce the variance of Monte Carlo estimators, but constructing effective control variates can be challenging when the number of samples is small. In this paper, we show that when a large number of related integrals need to be computed, it is possible to leverage the similarity between these integration tasks to improve performance even when the number of samples per task is very small. Our approach, called meta learning CVs (Meta-CVs), can be used for up to hundreds or thousands of tasks. Our empirical assessment indicates that Meta-CVs can lead to significant variance reduction in such settings, and our theoretical analysis establishes general conditions under which Meta-CVs can be successfully trained.

show abstract

Section: Control Variate Methodsmentioning

confidence: 99%

Meta-learning Control Variates: Variance Reduction with Limited Data

Sun¹,

Oates²,

Briol³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…This can be contrasted with the typically asymptotic analysis of MCMC. The practical estimation of the final term in this bound was discussed in Section 4 of South et al (2021).…”

Section: Methodsmentioning

confidence: 99%

Optimal Thinning of MCMC Output

Riabiz

Chen

Cockayne

et al. 2022

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

The use of heuristics to assess the convergence and compress the output of Markov chain Monte Carlo can be sub-optimal in terms of the empirical approximations that are produced. Typically a number of the initial states are attributed to 'burn in' and removed, while the remainder of the chain is 'thinned' if compression is also required. In this paper, we consider the problem of retrospectively selecting a subset of states, of fixed cardinality, from the sample path such that the approximation provided by their empirical distribution is close to optimal. A novel method is proposed, based on greedy minimisation of a kernel Stein discrepancy, that is suitable when the gradient of the log-target can be evaluated and approximation using a small number of states is required. Theoretical results guarantee consistency of the method and its effectiveness is demonstrated in the challenging context of parameter inference for ordinary differential equations. Software is available in the Stein Thinning package in Python, R and MATLAB.

show abstract

“…The main differences between existing gradient-based control variates are in how the function class H is specified. It is the class of Qth order polynomials in ZVCV (Assaraf and Caffarel, 1999;Mira et al, 2013), a reproducing kernel Hilbert space in control functionals (CF, Oates et al, 2017), and a combination of these parameteric and non-parametric methods in semi-exact control functionals (SECF, South et al, 2022a). More details about these methods are provided in Sections 1.4.1, 1.4.2 and 1.4.3, respectively.…”

Section: Stein-based Control Variatesmentioning

confidence: 99%

Regularized Zero-Variance Control Variates

et al. 2023

View full text Add to dashboard Cite

Zero-variance control variates (ZV-CV) are a post-processing method to reduce the variance of Monte Carlo estimators of expectations using the derivatives of the log target. Once the derivatives are available, the only additional computational effort lies in solving a linear regression problem. Significant variance reductions have been achieved with this method in low dimensional examples, but the number of covariates in the regression rapidly increases with the dimension of the target. In this paper, we present compelling empirical evidence that the use of penalized regression techniques in the selection of high-dimensional control variates provides performance gains over the classical least squares method. Another type of regularization based on using subsets of derivatives, or a priori regularization as we refer to it in this paper, is also proposed to reduce computational and storage requirements. Several examples showing the utility and limitations of regularized ZV-CV for Bayesian inference are given. The methods proposed in this paper are accessible through the R package ZVCV.

show abstract

Semi-exact control functionals from Sard’s method

Cited by 10 publications

References 35 publications

Meta-learning Control Variates: Variance Reduction with Limited Data

Meta-learning Control Variates: Variance Reduction with Limited Data

Optimal Thinning of MCMC Output

Regularized Zero-Variance Control Variates

Contact Info

Product

Resources

About