Quantifying uncertainty in variable selection with arbitrary matrices

Abstract

Probabilistically quantifying uncertainty in parameters, predictions and decisions is a crucial component of broad scientific and engineering applications. This is however difficult if the number of parameters far exceeds the sample size. Although there are currently many methods which have guarantees for problems characterized by large random matrices, there is often a gap between theory and practice when it comes to measures of statistical significance for matrices encountered in real-world applications. This paper proposes a scalable framework that utilizes state-of-the-art methods to provide approximations to the marginal posterior distributions. This framework is used to approximate marginal posterior inclusion probabilities for Bayesian variable selection.

Adaptive hierarchical priors for high-dimensional vector autoregressions

“…The third specification we consider for our hierarchical prior is the popular Spike-and-Slab prior, and follows very closely the approach described in van den Boom et al (2015a). While it is possible to cast this prior in the hierarchical form of (9) (see for example Gri n and Brown, 2010, p. 175), we follow the literature and write this prior as an explicit mixture of distributions…”

“…Also, unlike the approximation methods that rely on the restrictive natural conjugate prior (e.g., Banbura et al, 2010;Giannone et al, 2015), our suggested approach integrates hierarchical shrinkage within an independent prior setting. We capitalize on the e cient algorithm of van den Boom et al (2015a) designed for a univariate regression, and further develop it to address the complexities of high-dimensional VARs. In particular, we first rewrite the VAR in its fully recursive form, which allows equation-by-equation estimation (Carriero et al, 2017).…”

“…In particular, we first rewrite the VAR in its fully recursive form, which allows equation-by-equation estimation (Carriero et al, 2017). Next, as in van den Boom et al (2015a), estimation relies on a simple transformation ("rotation") of each VAR equation which allows to approximate the joint posterior of the VAR coe cients as the product of a number of scalar marginal posterior distributions. The algorithm, therefore, breaks the multivariate estimation problem into a series of independent tasks each one involving a scalar parameter.…”

“…The algorithm, therefore, breaks the multivariate estimation problem into a series of independent tasks each one involving a scalar parameter. 4 Finally, we extend and generalize the van den Boom et al (2015a) algorithm, originally developed to implement variable selection, to three popular cases of hierarchical priors, namely (i) Normal-Je↵reys (Hobert and Casella, 1996), (ii) Spike-and-Slab (Mitchell and Beauchamp, 1988), and (iii) Normal-Gamma (Gri n and Brown, 2010), and show how to obtain analytical posteriors for the VAR coe cients and the elements of the VAR covariance matrix.…”

Conditional central limit theorems for Gaussian projections

“…To upper bound the second term in (27) let (Z, Z E ) be defined according to Z = ΘX and Z E = ΘX E where the relationship between X and X E is given by (26). Conditioned on Θ and the event X ∈ E, we have Z ∼ P 2 and Z E ∼ P 3 , and thus…”