Convergence rates of posterior distributions

Bayesian Methods for Function Estimation

“…The resulting priors are recommended as default priors in infinite-dimensional spaces by Ghosal et al (1997). In Ghosal et al (2000), this idea was used with a spline basis for density estimation. They showed that with a suitable choice of k, depending on the sample size and the smoothness level of the target function, optimal convergence rates could be obtained.…”

“…Modifying the model to uniform(θ − 1, θ + 1), we see that the Kullback-Leibler numbers are infinite for every pair. Nevertheless, consistency for a general parametric family including such nonregular cases holds under continuity and positivity of the prior density at θ 0 provided that the general conditions of Ibragimov and Has'minskii (1981) can be verified; see Ghosal et al (1995) for details. For infinite-dimensional models, consistency may hold without Schwartz's condition on Kullback-Leibler support by exploiting special structure of the posterior distribution as in the case of the Dirichlet or a tail-free process.…”

“…Conditions for the rate of convergence given by Ghosal et al (2000) and described below are quantitative refinement of conditions for consistency. A similar result, but under a much stronger condition on bracketing entropy numbers, was given by Shen and Wasserman (2001).…”

“…Some variations of the theorem are given by Ghosal et al (2000), Ghosal and van der Vaart (2001) and Belitser and Ghosal (2003).…”

“…This large sample "merging of opinions" is equivalent to consistency (Blackwell and Dubins, 1962;Freedman, 1986a, 1986b;Ghosh et al, 1994). For virtually all finite-dimensional problems, the posterior distribution is consistent (Ibragimov and Has'minskii, 1981;Le Cam, 1986;Ghosal et al, 1995) if the prior does not rule out the true value. This is roughly a consequence of the fact that the likelihood is highly peaked near the true value of the parameter if the sample size is large.…”