The semi-parametric Bernstein-von Mises theorem for regression models with symmetric errors

Chae, Minwoo; Kim, Yongdai; Kleijn, B. J. K.

doi:10.5705/ss.202017.0074

Cited by 8 publications

(16 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We impose a product prior Π = Π Θ × Π H for (θ, η), where Π Θ and Π H are Borel probability measures on Θ = R p and H, respectively. We use a mixture of point masses at zero and continuous distributions for Π Θ , and a symmetrized DP mixture of normal distributions [9,10] for Π H .…”

Section: Priormentioning

confidence: 99%

“…We use a symmetrized DP mixture of normal prior for Π H , whose properties and inferential methods are well-known [9,10]…”

Section: Priormentioning

confidence: 99%

“…It can be shown [10] that v η = −P η 0 (l η ) for every η ∈ H 0 . Then, the Taylor expansion of L n (θ, η) around θ 0 roughly implies that…”

Section: Misspecified Lanmentioning

confidence: 99%

“…Asymptotic results for high-dimensional Bayesian model selection beyond Gaussian error are also quite novel. It should be noted that in contrast to current frequentist approaches, it is straightforward to modify computational algorithms developed for sparse linear models to allow unknown symmetric errors (see [9,24]).…”

Section: Introductionmentioning

confidence: 99%

“…These results rely on the assumption of Gaussian errors.Although some theoretical properties, such as consistency and rates of convergence, are robust to misspecification of η, methods that assume Gaussianity may still face many serious problems when η is non-Gaussian. First, although a point estimator may be consistent in nearly optimal rate, its efficiency is not satisfactory [10,23,42]. Also, confidence or credible sets do not provide correct uncertainty quantification under model misspecification [23].…”

mentioning

confidence: 99%

See 4 more Smart Citations

Bayesian sparse linear regression with unknown symmetric error

Chae

Lin

Dunson

2019

Information and Inference: A Journal of the IMA

Self Cite

View full text Add to dashboard Cite

We study full Bayesian procedures for sparse linear regression when errors have a symmetric but otherwise unknown distribution. The unknown error distribution is endowed with a symmetrized Dirichlet process mixture of Gaussians. For the prior on regression coefficients, a mixture of point masses at zero and continuous distributions is considered. We study behavior of the posterior with diverging number of predictors. Conditions are provided for consistency in the mean Hellinger distance. The compatibility and restricted eigenvalue conditions yield the minimax convergence rate of the regression coefficients in ℓ 1 -and ℓ 2 -norms, respectively. The convergence rate is adaptive to both the unknown sparsity level and the unknown symmetric error density under compatibility conditions. In addition, strong model selection consistency and a semi-parametric Bernstein-von Mises theorem are proven under slightly stronger conditions.Keywords: Adaptive contraction rates, Bernstein von-Mises theorem, Dirichlet process mixture, high-dimensional semiparametric model, sparse prior, symmetric error * This article has been accepted for publication in Information and Inference Published by Oxford University Press. The accepted version contains significantly improved results, which is available at https://doi.org/10.1093/imaiai/iay022 the high-dimensional setting where p, the number of the predictors and the size of the coefficient vector, may grow with the sample size n, and possibly p ≫ n. If p > n, model (1) is not identifiable due to the singularity of its design matrix, therefore θ is not estimable unless further restrictions or structures are imposed. A standard assumption for θ is the sparsity condition which assumes that most components of θ are zero. For the last two decades, model (1) has been extensively studied under various sparsity conditions, in particular through penalized regression approaches such as Lasso and its various variants or extensions [36,37,46,47]. Recent advances in MCMC and other computational algorithms have led to a growing development of Bayesian models incorporating sparse priors [7,8,13,20,27]. In general, two classes of sparse priors are often used, the first being the spike and slab type (see e.g., [7,8]), with some recent work extending to continuous versions [20,28,32,33], and the other being continuous shrinkage priors; in particular, local-global shrinkage priors (see [1,5,31]).In the literature, both frequentist and Bayesian, the standard Gaussian error model, in which ǫ i 's are assumed to be i.i.d. Gaussian, is typically adopted, providing substantial computational and theoretical benefits. Using a squared error loss function, various penalization techniques are developed. Theoretical aspects of such estimates have been explored, showing recovery of θ in nearly optimal rate or optimal selection of the true nonzero coefficients [3,7,11,12,21]. More recent theoretical advances assure that relying on certain desparsifying techniques, asymptotically optimal (or at least honest) confidence set...

show abstract

Section: Priormentioning

confidence: 99%

“…We use a symmetrized DP mixture of normal prior for Π H , whose properties and inferential methods are well-known [9,10]…”

Section: Priormentioning

confidence: 99%

“…It can be shown [10] that v η = −P η 0 (l η ) for every η ∈ H 0 . Then, the Taylor expansion of L n (θ, η) around θ 0 roughly implies that…”

Section: Misspecified Lanmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Bayesian sparse linear regression with unknown symmetric error

Chae

Lin

Dunson

2019

Information and Inference: A Journal of the IMA

Self Cite

View full text Add to dashboard Cite

show abstract

Semiparametric estimation for linear regression with symmetric errors

Chee

Seo

2020

Computational Statistics & Data Analysis

View full text Add to dashboard Cite

Large Sample Justifications for the Bayesian Empirical Likelihood

Sueishi

2022

Econom. Theory

View full text Add to dashboard Cite

This study investigates the asymptotic properties of the Bayesian empirical likelihood (BEL), which uses the empirical likelihood as an alternative to a parametric likelihood for Bayesian inference. We establish two asymptotic equivalence results based on the Bernstein–von Mises (BvM) theorem by introducing a new formulation of the moment restriction model. First, the limiting posterior distribution of the BEL is the same as that of a parametric Bayesian method that uses the likelihood of a least favorable model of the moment restriction model. Second, the limiting posterior distribution is also the same as that of a semiparametric Bayesian method that places priors on both a finite-dimensional parameter of interest and an infinite-dimensional nuisance parameter. Because parametric and semiparametric Bayesian methods are legitimate Bayesian procedures, the equivalence results provide a large sample justification for the BEL as a Bayesian inference method. Moreover, the BvM theorem provides a frequentist justification for BEL posterior inference.

show abstract

The semi-parametric Bernstein-von Mises theorem for regression models with symmetric errors

Cited by 8 publications

References 35 publications

Bayesian sparse linear regression with unknown symmetric error

Bayesian sparse linear regression with unknown symmetric error

Semiparametric estimation for linear regression with symmetric errors

Large Sample Justifications for the Bayesian Empirical Likelihood

Contact Info

Product

Resources

About