A comparison of learning rate selection methods in generalized Bayesian inference

Abstract

Generalized Bayes posterior distributions are formed by putting a fractional power on the likelihood before combining with the prior via Bayes's formula. This fractional power, which is often viewed as a remedy for potential model misspecification bias, is called the learning rate, and a number of data-driven learning rate selection methods have been proposed in the recent literature. Each of these proposals has a different focus, a different target they aim to achieve, which makes them difficult to compare. In this paper, we provide a direct head-to-head comparison of these learning rate selection methods in various misspecified model scenarios, in terms of several relevant metrics, in particular, coverage probability of the generalized Bayes credible regions. In some examples all the methods perform well, while in others the misspecification is too severe to be overcome, but we find that the so-called generalized posterior calibration algorithm tends to outperform the others in terms of credible region coverage probability.

Gibbs posterior inference on a Levy density under discrete sampling

“…While the asymptotic concentration results hold for any fixed choice of ω (with an appropriate corresponding choice of β in the context of Theorem 3), the fact that ω does affect the finite-sample performance means that data-driven choices are needed. As we briefly mentioned in Section 3, there is an active literature on data-dependent learning rate selection; see Wu and Martin (2020) for a comparison. However, it remains unclear how these various methods might perform in an especially challenging application such as this, especially one with an infinite-dimensional quantity of interest.…”

Calibrating generalized predictive distributions

“…The overall conclusion drawn in Wu and Martin (2020) is that only the GPC algorithm provides satisfactory calibration of the generalized posterior credible sets in general. This is not surprising, given that the other methods are designed to achieve other properties.…”

“…A number of different data-driven methods for selecting the learning rate η have been proposed in the recent literature. Here we give just a brief summary of these; a more thorough review can be found in Wu and Martin (2020).…”

“…Instead, η is a tuning parameter and must be chosen to achieve some desirable operating characteristic. A number of learning rate selection procedures have been developed in the literature recently, including Holmes and Walker (2017), Lyddon et al (2019), Grünwald and van Ommen (2017), and Syring and Martin (2019); see Wu and Martin (2020) for a comparison. For sure, whatever operating characteristics are achieved for the generalized posterior through these choices of η do not automatically carry over to the corresponding predictive distribution, so it is necessary to reconsider things when prediction is the goal.…”

Robust Generalised Bayesian Inference for Intractable Likelihoods

“…This provides a heuristic motivation for the default β = 1. However, in a misspecified setting smaller values of β are needed to avoid over-confidence in the generalised posterior, taking misspecification into account; see the recent review of Wu and Martin (2020). Here we aim to pick β such that the scale of the asymptotic covariance matrix of the generalised posterior (H −1 * ; Theorem 2) matches that of the minimum MMD point estimator (H −1 * J * H −1 * ; Lemma 4), an approach proposed in Lyddon et al (2019).…”