A Comparison of Learning Rate Selection Methods in Generalized Bayesian Inference

Wu, Pei-Shien; Martin, Ryan

doi:10.1214/21-ba1302

Cited by 12 publications

(8 citation statements)

References 41 publications

(57 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This provides a heuristic motivation for the default β = 1. However, in a misspecified setting smaller values of β are needed to avoid over‐confidence in the generalised posterior, taking misspecification into account; see the recent review of Wu and Martin (2020). Here we aim to pick β such that the scale of the asymptotic precision matrix of the generalised posterior (

H_{*}

; Theorem 2) matches that of the minimum KSD point estimator (

H_{*} J_{*}^{- 1} H_{*}

; Lemma 4), an approach proposed in Lyddon et al (2019).…”

Section: Default Settings For Ksd‐bayesmentioning

confidence: 99%

Robust Generalised Bayesian Inference for Intractable Likelihoods

Matsubara

Knoblauch

Briol

et al. 2022

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

Generalised Bayesian inference updates prior beliefs using a loss function, rather than a likelihood, and can therefore be used to confer robustness against possible mis‐specification of the likelihood. Here we consider generalised Bayesian inference with a Stein discrepancy as a loss function, motivated by applications in which the likelihood contains an intractable normalisation constant. In this context, the Stein discrepancy circumvents evaluation of the normalisation constant and produces generalised posteriors that are either closed form or accessible using the standard Markov chain Monte Carlo. On a theoretical level, we show consistency, asymptotic normality, and bias‐robustness of the generalised posterior, highlighting how these properties are impacted by the choice of Stein discrepancy. Then, we provide numerical experiments on a range of intractable distributions, including applications to kernel‐based exponential family models and non‐Gaussian graphical models.

show abstract

H_{*}

; Theorem 2) matches that of the minimum KSD point estimator (

H_{*} J_{*}^{- 1} H_{*}

; Lemma 4), an approach proposed in Lyddon et al (2019).…”

Section: Default Settings For Ksd‐bayesmentioning

confidence: 99%

Robust Generalised Bayesian Inference for Intractable Likelihoods

Matsubara

Knoblauch

Briol

et al. 2022

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

show abstract

“…There are various approaches to estimation of η that lead to the posterior distribution concentrating at θ ⋆ , e.g. [17] and [31]; see a review [45]. Here I will give a very brief discussion.…”

Section: Estimation Of ηmentioning

confidence: 99%

Bernstein - von Mises theorem and misspecified models: a review

Bochkina¹

2022

Preprint

View full text Add to dashboard Cite

IntroductionConsider a family of probability models P (Y | θ) indexed by parameter θ ∈ Θ for observations y, and a prior distribution π on the parameter space Θ. In a classical Bayesian approach, the posterior distributionDenote the true distribution of observations P 0 , and we consider the casethe model is misspecified. Such case arises in many applications, particularly in complex models where the numerical evaluation of posterior distribution under the ideal probability model takes a long time to run, leading to increased use of approximate models with faster computing time. A typical example is approximating complex dependence structure by pairwise dependence only [2], [40].For well-specified regular models, the classical Bernstein -von Mises theorem states that for n independent identically distributed (iid) observations, for large n, the posterior distribution behaves approximately as a normal distribution centered on the true value of the parameter with a random shift, with both the posterior variance and the variance of the random shift being asymptotically equal to the inverse Fisher information, thus making the Bayesian inference asymptotically consistent and efficient.This was extended to locally asymptotically normal (LAN) models for Θ ∈ R p with fixed p [39], for LAN models with growing p, etc.A version of Bernstein -von Mises theorem is also available for nonregular models such as locally asymptotically exponential models ([20], [9]), models with parameter on the boundary of the parameter space [7] which hold under misspecified models. Here, however, we focus on "regular" models where estimators are asymptotically Gaussian.However, under model misspecification, Bayesian model is no longer optimal, as the posterior variance does not match the minimal lower bound on the variance of unbiased estimators [22] and [29]).Therefore, the standard way of constructing a posterior distribution following the Bayes theorem may not be appropriate for particular purposes, e.g. inference or prediction [27]. Different ways to construct a distribution of θ given Y that produces inference appropriate for the purpose of the analysis have been proposed. A natural aim for such a method is to behave like a standard Bayesian method when the

show abstract

“…At η = 1 η-SMI is standard Bayesian inference and at η = 0 η-SMI reproduces the Cut-model. Carmona and Nicholls (2020) suggest choosing η maximizing the expected log pointwise predictive density (ELPD) though this choice is not an essential part of their method and other criteria (Wu and Martin, 2020) may be more appropriate in different settings. Liu and Goudie (2021) adapt η-SMI for Geographically Weighted Regression using an influence parameter across like-lihood factors which is modeled as a function of distance between the spatial observation locations.…”

Section: Introductionmentioning

confidence: 99%

Valid belief updates for prequentially additive loss functions arising in Semi-Modular Inference

Nicholls¹,

Lee²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Model-based Bayesian evidence combination leads to models with multiple parameteric modules. In this setting the effects of model misspecification in one of the modules may in some cases be ameliorated by cutting the flow of information from the misspecified module. Semi-Modular Inference (SMI) is a framework allowing partial cuts which modulate but do not completely cut the flow of information between modules. We show that SMI is part of a family of inference procedures which implement partial cuts. It has been shown that additive losses determine an optimal, valid and order-coherent belief update. The losses which arise in Cut models and SMI are not additive. However, like the prequential score function, they have a kind of prequential additivity which we define. We show that prequential additivity is sufficient to determine the optimal valid and order-coherent belief update and that this belief update coincides with the belief update in each of our SMI schemes.

show abstract

A Comparison of Learning Rate Selection Methods in Generalized Bayesian Inference

Cited by 12 publications

References 41 publications

Robust Generalised Bayesian Inference for Intractable Likelihoods

Robust Generalised Bayesian Inference for Intractable Likelihoods

Bernstein - von Mises theorem and misspecified models: a review

Valid belief updates for prequentially additive loss functions arising in Semi-Modular Inference

Contact Info

Product

Resources

About