High-dimensional multivariate posterior consistency under global–local shrinkage priors

Abstract

We consider sparse Bayesian estimation in the classical multivariate linear regression model with p regressors and q response variables. In univariate Bayesian linear regression with a single response y, shrinkage priors which can be expressed as scale mixtures of normal densities are popular for obtaining sparse estimates of the coefficients. In this paper, we extend the use of these priors to the multivariate case to estimate a p × q coefficients matrix B. We derive sufficient conditions for posterior consistency under the Bayesian multivariate linear regression framework and prove that our method achieves posterior consistency even when p > n and even when p grows at nearly exponential rate with the sample size. We derive an efficient Gibbs sampling algorithm and provide the implementation in a comprehensive R package called MBSP. Finally, we demonstrate through simulations and data analysis that our model has excellent finite sample performance.

Bayesian variable selection for multioutcome models through shared shrinkage

“…Following the arguments from Schwartz (1965) (also used in Bai and Ghosh (2018) and Armagan et al (2013)), the first condition in Theorem 2 can be shown to imply Φ n → 0 a.s. Further, e nC J 1n → 0 a.s., for a constant C > 0 that may depend on auxiliary parameters (such as Ψ and the eigenvalues of the design matrix) but not on B 0 . Similarly, the second condition of the Theorem 2 can be shown to imply that for any constant c > 0, e nc J 2n → ∞ a.s.…”

“…This relaxation comes at a cost, mainly assumption (A2), which essentially bounds the entries of the design matrix. In contrast, Bai and Ghosh (2018) assume upper and lower (asymptotic) bounds on the eigenvalues of the design matrix. However, the flexibility gained under our choice is significant, as we require no condition (except continuity) on the prior for B.…”

“…However, the flexibility gained under our choice is significant, as we require no condition (except continuity) on the prior for B. This is much more general than the assumptions made in the consistency theorems of Bai and Ghosh (2018) and Armagan et al (2013). Their choices require conditions on the prior with convoluted formulas involving Ψ and the eigenvalues of the design matrix, thus restricting the choice of prior on B in ways that are not straightforward.…”

“…• MBSP Model (Bai and Ghosh, 2018): As previously noted, this approach is similar to our MOHS model where the global parameter τ is common across all responses and fixed by asymptotic arguments, rather than estimated from the data. The performance of this model is obtained using their available R package MBSP.…”

Novel Bayesian methodology in multivariate problems.

“…Hence, in this work we focus on developing a more exible variable selection method that encourages the inclusion of similar sets of predictors in each of the K models by extending the GL shrinkage framework. Recently, Bai and Ghosh (2018) independently explored a similar setup and proposed their Multivariate Bayesian Model with Shrinkage Priors (MBSP). We will discuss dierences that distinguish our work in later sections.…”