Spike and slab variable selection: Frequentist and Bayesian strategies

Ishwaran, Hemant; Rao, J. Sunil

doi:10.1214/009053604000001147

Cited by 817 publications

(714 citation statements)

References 38 publications

Supporting

Mentioning

710

Contrasting

Unclassified

Order By: Relevance

“…Based on the data, regression coefficients close to zero will be assigned to the spike, resulting in shrinkage towards 0, while coefficients that deviate substantially from zero will be assigned to the slab, resulting in (almost) no shrinkage. Early proposals of mixture priors can be found in George and McCulloch (1993) and Mitchell and Beauchamp (1988), and a scale mixture of normals formulation can be found in Ishwaran and Rao (2005). We will consider the following specification of the mixture prior:…”

Section: Discrete Normal Mixturementioning

confidence: 99%

“…Due to these advantages, Bayesian penalization is becoming increasingly popular in the literature (see e.g., Alhamzawi et al, 2012;Andersen et al, 2017;Armagan et al, 2013;Bae and Mallick, 2004;Bhadra et al, 2016;Bhattacharya et al, 2012;Bornn et al, 2010;Caron and Doucet, 2008;Carvalho et al, 2010;Feng et al, 2017;Griffin and Brown, 2017;Hans, 2009;Ishwaran and Rao, 2005;Lu et al, 2016;Peltola et al, 2014;Polson and Scott, 2011;Roy and Chakraborty, 2016;Zhao et al, 2016). An active area of research investigates theoretical properties of priors for Bayesian penalization, such as the Bayesian lasso prior (for a recent overview, see Bhadra et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Shrinkage priors for Bayesian penalized regression.

Erp¹,

Oberski²,

Mulder³

2018

Preprint

View full text Add to dashboard Cite

In linear regression problems with many predictors, penalized regression techniques are often used to guard against overfitting and to select variables relevant for predicting an outcome variable. Recently, Bayesian penalization is becoming increasingly popular in which the prior distribution performs a function similar to that of the penalty term in classical penalization. Specifically, the so-called shrinkage priors in Bayesian penalization aim to shrink small effects to zero while maintaining true large effects. Compared to classical penalization techniques, Bayesian penalization techniques perform similarly or sometimes even better, and they offer additional advantages such as readily available uncertainty estimates, automatic estimation of the penalty parameter, and more flexibility in terms of penalties that can be considered. However, many different shrinkage priors exist and the available, often quite technical, literature primarily focuses on presenting one shrinkage prior and often provides comparisons with only one or two other shrinkage priors. This can make it difficult for researchers to navigate through the many prior options and choose a shrinkage prior for the problem at hand. Therefore, the aim of this paper is to provide a comprehensive overview of the literature on Bayesian penalization. We provide a theoretical and conceptual comparison of nine different shrinkage priors and parametrize the priors, if possible, in terms of scale mixture of normal distributions to facilitate comparisons. We illustrate different characteristics and behaviors of the shrinkage priors and compare their performance in terms of prediction and variable selection in a simulation study. Additionally, we provide two empirical examples to illustrate the application of Bayesian penalization. Finally, an R package bayesreg is available online (https://github.com/sara-vanerp/bayesreg) which allows researchers to perform Bayesian penalized regression with novel shrinkage priors in an easy manner.

show abstract

Section: Discrete Normal Mixturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Shrinkage priors for Bayesian penalized regression.

Erp¹,

Oberski²,

Mulder³

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…BayesB is equivalent to the stochastic search variable selection method, which was originally developed by George and McCulloch (1993) and has been applied to QTL mapping by Yi et al (2003) and Wang et al (2005). BayesCp is still the stochastic search variable selection method with variable p and has been used by Ishwaran and Rao (2005) (who named it the spike and slab variable selection) and Xu (2007). From these points of view, the three 'BayesT' methods (BayesTA, BayesTB and BayesTCp) proposed herein may also be regarded as thresholdmodel-versions of the Bayesian shrinkage method and the stochastic search variable selection method.…”

Section: Common Data Set Of the Fourteenth Qtl-mas Workhopmentioning

confidence: 99%

Bayesian methods for estimating GEBVs of threshold traits

Wang

et al. 2012

Heredity

View full text Add to dashboard Cite

Estimation of genomic breeding values is the key step in genomic selection (GS). Many methods have been proposed for continuous traits, but methods for threshold traits are still scarce. Here we introduced threshold model to the framework of GS, and specifically, we extended the three Bayesian methods BayesA, BayesB and BayesCp on the basis of threshold model for estimating genomic breeding values of threshold traits, and the extended methods are correspondingly termed BayesTA, BayesTB and BayesTCp. Computing procedures of the three BayesT methods using Markov Chain Monte Carlo algorithm were derived. A simulation study was performed to investigate the benefit of the presented methods in accuracy with the genomic estimated breeding values (GEBVs) for threshold traits. Factors affecting the performance of the three BayesT methods were addressed. As expected, the three BayesT methods generally performed better than the corresponding normal Bayesian methods, in particular when the number of phenotypic categories was small. In the standard scenario (number of categories ¼ 2, incidence ¼ 30%, number of quantitative trait loci ¼ 50, h 2 ¼ 0.3), the accuracies were improved by 30.4%, 2.4%, and 5.7% points, respectively. In most scenarios, BayesTB and BayesTCp generated similar accuracies and both performed better than BayesTA. In conclusion, our work proved that threshold model fits well for predicting GEBVs of threshold traits, and BayesTCp is supposed to be the method of choice for GS of threshold traits.

show abstract

“…A normal posterior simplifies the MCMC sampling process because the Gibbs sampler can be used to draw the regression coefficient. Other prior distributions have been proposed, e.g., the mixture prior of two normal distributions (George and Mcmulloch 1993;Yi et al 2003) and the spike and slab model (Ishwaran and Rao 2005). A t-distribution may also be used as a prior for the regression coefficient.…”

Section: Discussionmentioning

confidence: 99%

Derivation of the Shrinkage Estimates of Quantitative Trait Locus Effects

2007

Genetics

View full text Add to dashboard Cite

The shrinkage estimate of a quantitative trait locus (QTL) effect is the posterior mean of the QTL effect when a normal prior distribution is assigned to the QTL. This note gives the derivation of the shrinkage estimate under the multivariate linear model. An important lemma regarding the posterior mean of a normal likelihood combined with a normal prior is introduced. The lemma is then used to derive the Bayesian shrinkage estimates of the QTL effects.

show abstract

Spike and slab variable selection: Frequentist and Bayesian strategies

Cited by 817 publications

References 38 publications

Shrinkage priors for Bayesian penalized regression.

Shrinkage priors for Bayesian penalized regression.

Bayesian methods for estimating GEBVs of threshold traits

Derivation of the Shrinkage Estimates of Quantitative Trait Locus Effects

Contact Info

Product

Resources

About