Bernstein - von Mises theorem and misspecified models: a review

Abstract

IntroductionConsider a family of probability models P (Y | θ) indexed by parameter θ ∈ Θ for observations y, and a prior distribution π on the parameter space Θ. In a classical Bayesian approach, the posterior distributionDenote the true distribution of observations P 0 , and we consider the casethe model is misspecified. Such case arises in many applications, particularly in complex models where the numerical evaluation of posterior distribution under the ideal probability model takes a long time to run, leading to increased use of approximate models with faster computing time. A typical example is approximating complex dependence structure by pairwise dependence only [2], [40].For well-specified regular models, the classical Bernstein -von Mises theorem states that for n independent identically distributed (iid) observations, for large n, the posterior distribution behaves approximately as a normal distribution centered on the true value of the parameter with a random shift, with both the posterior variance and the variance of the random shift being asymptotically equal to the inverse Fisher information, thus making the Bayesian inference asymptotically consistent and efficient.This was extended to locally asymptotically normal (LAN) models for Θ ∈ R p with fixed p [39], for LAN models with growing p, etc.A version of Bernstein -von Mises theorem is also available for nonregular models such as locally asymptotically exponential models ([20], [9]), models with parameter on the boundary of the parameter space [7] which hold under misspecified models. Here, however, we focus on "regular" models where estimators are asymptotically Gaussian.However, under model misspecification, Bayesian model is no longer optimal, as the posterior variance does not match the minimal lower bound on the variance of unbiased estimators [22] and [29]).Therefore, the standard way of constructing a posterior distribution following the Bayes theorem may not be appropriate for particular purposes, e.g. inference or prediction [27]. Different ways to construct a distribution of θ given Y that produces inference appropriate for the purpose of the analysis have been proposed. A natural aim for such a method is to behave like a standard Bayesian method when the

Generalised Bayesian Inference for Discrete Intractable Likelihood

“…The result was established using similar arguments from early work by Hooker and Vidyashankar (2014); Ghosh and Basu (2016) and extended techniques of Miller (2021); Matsubara et al (2022). See also the recent review of Bochkina (2022).…”

Normal approximation for the posterior in exponential families

“…Remark 1. As λ(w) = nβ(w) + η(w), in 'typical' situations the row sums in (7) are thus of order n −1/2 , and we infer that the bounds of Theorem 1 are of order n −1/2 for large n. A simple sufficient (but not necessary) condition under which this occurs is that β does not depend on the sample size n and that ∂ u,v β( θ) = 0 for all 1 ≤ u, v ≤ k.…”

“…All the above expressions simplify in the single parameter (k = 1) case: the conditions from Assumptions A or L A become transparent, the scalars from (7)…”

“…Within the vast literature on the BvM Theorem, we single out the following topics that are of current interest: BvM theorems for misspecified models, see [7]; semi-parametric BvM theorems, see [6]; high dimensional settings when the dimension of the parameter space increases with sample size, see [33]. A common trait of many of these questions is the need for non-asymptotic (i.e.…”