Learning from a lot: Empirical Bayes for high‐dimensional model‐based prediction

Wiel, Mark A. van de; Beest, Dennis E. te; Münch, Magnus

doi:10.1111/sjos.12335

Cited by 18 publications

(15 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another aspect is that "pile-up" effects can be avoided by choosing a small hotspot propensity variance at the risk of giving up substantial hotspot selection performance. The very sparse nature of molecular QTL analyses also rules out the use of simple empirical Bayes estimates which typically collapse to the degenerate caseσ 2 ω = 0; see, for example, Scott and Berger (2010), van de Wiel, Te Beest and Münch (2019). Thus, a tailored solution is needed.…”

Section: Problem Statementmentioning

confidence: 99%

A global-local approach for detecting hotspots in multiple-response regression

Ruffieux¹,

Davison²,

Hager³

et al. 2020

Ann. Appl. Stat.

View full text Add to dashboard Cite

We tackle modelling and inference for variable selection in regression problems with many predictors and many responses. We focus on detecting hotspots, that is, predictors associated with several responses. Such a task is critical in statistical genetics, as hotspot genetic variants shape the architecture of the genome by controlling the expression of many genes and may initiate decisive functional mechanisms underlying disease endpoints. Existing hierarchical regression approaches designed to model hotspots suffer from two limitations: their discrimination of hotspots is sensitive to the choice of top-level scale parameters for the propensity of predictors to be hotspots, and they do not scale to large predictor and response vectors, for example, of dimensions 10 3 -10 5 in genetic applications. We address these shortcomings by introducing a flexible hierarchical regression framework that is tailored to the detection of hotspots and scalable to the above dimensions. Our proposal implements a fully Bayesian model for hotspots based on the horseshoe shrinkage prior. Its global-local formulation shrinks noise globally and, hence, accommodates the highly sparse nature of genetic analyses while being robust to individual signals, thus leaving the effects of hotspots unshrunk. Inference is carried out using a fast variational algorithm coupled with a novel simulated annealing procedure that allows efficient exploration of multimodal distributions.

show abstract

Section: Problem Statementmentioning

confidence: 99%

A global-local approach for detecting hotspots in multiple-response regression

Ruffieux¹,

Davison²,

Hager³

et al. 2020

Ann. Appl. Stat.

View full text Add to dashboard Cite

show abstract

“…The intuition behind empirical Bayes and cross-validation is similar: empirical Bayes aims to choose the value for λ that is best in predicting the full data set, while cross-validation aims to choose the value for λ that is best in predicting the validation set given a training set. A possible disadvantage of empirical Bayes and cross-validation is that the (marginal) likelihood can be flat or multimodal when there are multiple penalty parameters (van de Wiel et al, 2017). 4 Throughout this paper, we will focus on the full and empirical Bayes approach to determine λ, and only consider cross-validation for the frequentist penalization methods we will compare the priors to.…”

Section: Bayesian Penalized Regressionmentioning

confidence: 99%

“…Empirical Bayes. Empirical Bayes (EB) methods, also known as the "evidence" procedure (see e.g., Wolpert and Strauss, 1996), first estimate the penalty parameter λ from the data and then plug in this EB estimate for λ in the model (see van de Wiel et al, 2017, for an overview of EB methodology in high-dimensional data). The resulting prior is called an EB prior.…”

Section: Bayesian Penalized Regressionmentioning

confidence: 99%

Shrinkage priors for Bayesian penalized regression.

Erp¹,

Oberski²,

Mulder³

2018

Preprint

View full text Add to dashboard Cite

In linear regression problems with many predictors, penalized regression techniques are often used to guard against overfitting and to select variables relevant for predicting an outcome variable. Recently, Bayesian penalization is becoming increasingly popular in which the prior distribution performs a function similar to that of the penalty term in classical penalization. Specifically, the so-called shrinkage priors in Bayesian penalization aim to shrink small effects to zero while maintaining true large effects. Compared to classical penalization techniques, Bayesian penalization techniques perform similarly or sometimes even better, and they offer additional advantages such as readily available uncertainty estimates, automatic estimation of the penalty parameter, and more flexibility in terms of penalties that can be considered. However, many different shrinkage priors exist and the available, often quite technical, literature primarily focuses on presenting one shrinkage prior and often provides comparisons with only one or two other shrinkage priors. This can make it difficult for researchers to navigate through the many prior options and choose a shrinkage prior for the problem at hand. Therefore, the aim of this paper is to provide a comprehensive overview of the literature on Bayesian penalization. We provide a theoretical and conceptual comparison of nine different shrinkage priors and parametrize the priors, if possible, in terms of scale mixture of normal distributions to facilitate comparisons. We illustrate different characteristics and behaviors of the shrinkage priors and compare their performance in terms of prediction and variable selection in a simulation study. Additionally, we provide two empirical examples to illustrate the application of Bayesian penalization. Finally, an R package bayesreg is available online (https://github.com/sara-vanerp/bayesreg) which allows researchers to perform Bayesian penalized regression with novel shrinkage priors in an easy manner.

show abstract

“…A Bayesian could argue that endowment of

α

with a hyperprior results in propagation of uncertainty about

α

and as a result improved regression parameter uncertainty quantification. Van de Wiel, te Beest, and Münch (2019) show in a similar setting that EB estimation of hyperparameters does not necessarily lead to worse uncertainty quantification as measured by frequentist coverage of Bayesian credible intervals, as compared to a full Bayes treatment of the hyperparameters.…”

Section: Discussionmentioning

confidence: 97%

Drug sensitivity prediction with normal inverse Gaussian shrinkage informed by external data

et al. 2020

Self Cite

View full text Add to dashboard Cite

In precision medicine, a common problem is drug sensitivity prediction from cancer tissue cell lines. These types of problems entail modelling multivariate drug responses on high‐dimensional molecular feature sets in typically >1000 cell lines. The dimensions of the problem require specialised models and estimation methods. In addition, external information on both the drugs and the features is often available. We propose to model the drug responses through a linear regression with shrinkage enforced through a normal inverse Gaussian prior. We let the prior depend on the external information, and estimate the model and external information dependence in an empirical‐variational Bayes framework. We demonstrate the usefulness of this model in both a simulated setting and in the publicly available Genomics of Drug Sensitivity in Cancer data.

show abstract

Learning from a lot: Empirical Bayes for high‐dimensional model‐based prediction

Cited by 18 publications

References 65 publications

A global-local approach for detecting hotspots in multiple-response regression

A global-local approach for detecting hotspots in multiple-response regression

Shrinkage priors for Bayesian penalized regression.

Drug sensitivity prediction with normal inverse Gaussian shrinkage informed by external data

Contact Info

Product

Resources

About