Bayesian Optimal Adaptive Estimation Using a Sieve Prior

Abstract

We derive rates of contraction of posterior distributions on nonparametric models resulting from sieve priors. The aim of the paper is to provide general conditions to get posterior rates when the parameter space has a general structure, and rate adaptation when the parameter space is, e.g., a Sobolev class. The conditions employed, although standard in the literature, are combined in a different way. The results are applied to density, regression, nonlinear autoregression and Gaussian white noise models. In the latter we have also considered a loss function which is different from the usual l 2 norm, namely the pointwise loss. In this case it is possible to prove that the adaptive Bayesian approach for the l 2 loss is strongly suboptimal and we provide a lower bound on the rate.

Bayes procedures for adaptive inference in inverse problems for the white noise model

“…This shows that M n is positive throughout (0, α n + 1/ log n] with probability tending to one uniformly over 2 . Since E 0 Y 2 i = κ 2 i μ 2 0,i + 1/n, the expected value on the left-hand side of (5.7) is equal to…”

“…Lemma 7(i) further bounds the right hand side of the above display by a multiple of n −1/(1+2α+2 p) (log n) 2 (5.12). By Lemma 10, the second term in (5.12) is bounded by…”

“…By Lemma 1, β/4 < α n − 1/ log n for large enough n, both for the case that μ 0 ∈ S β and μ 0 ∈ A γ , so this interval is included in (β/4, ∞). Taking c = β/2 + 2 p in Lemma 7(i) then shows that the first term is bounded by a multiple of 1/(log n) 2 and hence tends to zero, uniformly over [α n − 1/ log n, α n ]. We now consider the second term in (5.4), which is equal to h n (α)/(log n) 2 .…”

“…Taking c = β/2 + 2 p in Lemma 7(i) then shows that the first term is bounded by a multiple of 1/(log n) 2 and hence tends to zero, uniformly over [α n − 1/ log n, α n ]. We now consider the second term in (5.4), which is equal to h n (α)/(log n) 2 . By Lemma 2 for any μ 0 ∈ 2 and n ≥ e 4 we have…”

“…In the paper [5] it is shown that in this case adaptive Bayesian inference is possible using a hierarchical, conditionally Gaussian prior, while in [35] partially adaptation is shown using Gaussian priors with scale parameter determined by an empirical Bayes method. Other recent papers also exhibit priors that yield rate-adaptive procedures in the direct signal-in-white-noise problem (see for instance [2,13,32,38]), but it is important to note that these papers use general theorems on contraction rates for posterior distributions (as given in [18] for instance) that are not suitable to deal with the truly ill-posed case in which k i → 0 as i → ∞. The reason is that if these general theorems are applied in the inverse case, we only obtain convergence rates relative to the (squared) norm μ → κ 2 i μ 2 i , which is not very interesting.…”