Convergence rates for posterior distributions and adaptive estimation

Abstract

The goal of this paper is to provide theorems on convergence rates of posterior distributions that can be applied to obtain good convergence rates in the context of density estimation as well as regression. We show how to choose priors so that the posterior distributions converge at the optimal rate without prior knowledge of the degree of smoothness of the density function or the regression function to be estimated.Comment: Published by the Institute of Mathematical Statistics (http://www.imstat.org) in the Annals of Statistics (http://www.imstat.org/aos/) at http://dx.doi.org/10.1214/00905360400000049

Bayesian regression with nonparametric heteroskedasticity

“…Posterior convergence rates were studied in Ghosal et al (2000) and Shen and Wasserman (2001). Belitser and Ghosal (2003), Ghosal et al (2003), Huang (2004) the theorem implies asymptotic equivalence between standard confidence and credible sets. Chernozhukov and Hong (2003) and Kleijn and van der Vaart (2012) prove Bernstein-von Mises type theorems for misspecified parametric models, 2 and Panov and Spokoiny (2013) consider settings with increasing dimension of the nuisance parameter.…”

Bayesian Methods for Function Estimation

“…Belitser and Ghosal (2003) showed that the strategy works for an infinite-dimensional normal. Ghosal et al (2003) and Huang (2004) obtained similar results for the density estimation problem. Kleijn and van der Vaart (2002) considered the issue of misspecification, where p 0 may not lie in the support of the prior.…”

Asymptotic properties of posterior distributions in nonparametric regression with non-Gaussian errors

“…For these purposes, we consider only the case in which the support of the prior distribution contains only uniformly bounded regression functions as stated in the Assumption P.3 below. Similar assumptions about the uniform boundedness of regression function can be found in Shen and Wasserman (2001) and Huang (2004). In this case, the probability distribution of Q is assumed to be known.…”

“…Posterior consistency and the question about the rate of convergence of posterior distribution in nonparametric regression problems have been mainly studied under Gaussian noise distribution (e.g. Shen and Wasserman 2001;Huang 2004;Choi and Schervish 2007) and further efforts are expected to be taken under the general noise distribution.…”