On Bayesian Oracle Properties

Abstract

When model uncertainty is handled by Bayesian model averaging (BMA) or Bayesian model selection (BMS), the posterior distribution possesses a desirable "oracle property" for parametric inference, if for large enough data it is nearly as good as the oracle posterior, obtained by assuming unrealistically that the true model is known and only the true model is used. We study the oracle properties in a very general context of quasi-posterior, which can accommodate non-regular models with cubic root asymptotics and partial identification. Our approach for proving the oracle properties is based on a unified treatment that bounds the posterior probability of model mis-selection. This theoretical framework can be of interest to Bayesian statisticians who would like to theoretically justify their new model selection or model averaging methods in addition to empirical results. Furthermore, for non-regular models, we obtain nontrivial conclusions on the choice of prior penalty on model complexity, the temperature parameter of the quasi-posterior, and the advantage of BMA over BMS.

Bayesian Fixed-domain Asymptotics: Bernstein-von Mises Theorem for Covariance Parameters in a Gaussian Process Model

“…In the Bayesian setup, the asymptotic property of posterior distributions in partially identified models have been studied in Moon and Schorfheide [51], Gustafson [29], Jiang [34], Chen et al [15], Jiang and Li [35], etc. In particular, Moon and Schorfheide [51] proves that the posterior distribution of those parameters that can be identified from the data (called "reduced-form parameters", θ in our model) has the standard n −1/2 -convergence to a normal limit, and the posterior of the full partially identified parameter vector (called "structural parameters", (θ, α) in our model) converges to the conditional prior given the MLE of the reduced-form parameter.…”