Bayesian adaptive lasso with variational Bayes for variable selection in high-dimensional generalized linear mixed models

Tung, Dao Thanh; Tran, Minh‐Ngoc; Cuong, Tran Manh

doi:10.1080/03610918.2017.1387663

Cited by 7 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the Bayesian adaptive lasso (BaLasso) prior proposed by [45] it is also possible to extend our individual fixed-effects selection approach to account for (ordered) group selection by imposing different shrinkage levels to different coefficients. A variational Bayes approach for generalized linear models with priors of this type has been explored by [81], where the variational parameter updates of their VBGLMM algorithm could be streamlined using the framework studied in our work.…”

Section: Discussionmentioning

confidence: 99%

“…[1] propose a sparse variational Bayes analysis of linear mixed models which focuses on random effects shrinkage via decomposition of the random effects vector covariance matrix. A more recent contribution is [81], where the suggested approach performs simultaneous fixed-effect selection and parameter estimation via variational Bayes and Bayesian adaptive lasso. However, the approach is limited to high-dimensional two-level generalized linear mixed models and does not account for any streamlined updating improvements.…”

Section: Contribution and Article Organizationmentioning

confidence: 99%

See 1 more Smart Citation

Sparse linear mixed model selection via streamlined variational Bayes

Degani

Maestrini

Toczydlowska³

et al. 2022

Electron. J. Statist.

View full text Add to dashboard Cite

Linear mixed models are a versatile statistical tool to study data by accounting for fixed effects and random effects from multiple sources of variability. In many situations, a large number of candidate fixed effects is available and it is of interest to select a parsimonious subset of those being effectively relevant for predicting the response variable. Variational approximations facilitate fast approximate Bayesian inference for the parameters of a variety of statistical models, including linear mixed models. However, for models having a high number of fixed or random effects, simple application of standard variational inference principles does not lead to fast approximate inference algorithms, due to the size of model design matrices and inefficient treatment of sparse matrix problems arising from the required approximating density parameters updates.We illustrate how recently developed streamlined variational inference procedures can be generalized to make fast and accurate inference for the parameters of linear mixed models with nested random effects and globallocal priors for Bayesian fixed effects selection. Our variational inference algorithms achieve convergence to the same optima of their standard implementations, although with significantly lower computational effort, memory usage and time, especially for large numbers of random effects. Using simulated and real data examples, we assess the quality of automated procedures for fixed effects selection that are free from hyperparameters tuning and only rely upon variational posterior approximations. Moreover, we show high accuracy of variational approximations against model fitting via Markov Chain Monte Carlo sampling.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Contribution and Article Organizationmentioning

confidence: 99%

Sparse linear mixed model selection via streamlined variational Bayes

Degani

Maestrini

Toczydlowska³

et al. 2022

Electron. J. Statist.

View full text Add to dashboard Cite

show abstract

“…The penalization in model ( 2) is a particular case of the Bayesian lasso of Park and Casella (2008) that makes use of a Gamma prior on the Laplace squared scale parameter. Tung et al (2019) show the use of MFVB for variable selection in generalized linear mixed models via Bayesian adaptive Lasso. Ormerod et al (2017) develop a MFVB approximation to a linear model with a spike-and-slab prior on the regression coefficients.…”

Section: Alternative Response and Penalization Distributionsmentioning

confidence: 99%

A Variational Inference Framework for Inverse Problems

Maestrini¹,

Aykroyd²,

Wand³

2021

Preprint

View full text Add to dashboard Cite

We present a framework for fitting inverse problem models via variational Bayes approximations. This methodology guarantees flexibility to statistical model specification for a broad range of applications, good accuracy performances and reduced model fitting times, when compared with standard Markov chain Monte Carlo methods. The message passing and factor graph fragment approach to variational Bayes we describe facilitates streamlined implementation of approximate inference algorithms and forms the basis to software development. Such approach allows for supple inclusion of numerous response distributions and penalizations into the inverse problem model. Albeit our analysis is circumscribed to one-and two-dimensional response variables, we lay down an infrastructure where streamlining algorithmic steps based on nullifying weak interactions between variables are extendible to inverse problems in higher dimensions. Image processing applications motivated by biomedical and archaeological problems are included as illustrations.

show abstract

“…Armagan & Dunson (2011) propose a sparse variational Bayes analysis of linear mixed models which focuses on random effects shrinkage via decomposition of the random effects vector covariance matrix. A more recent contribution is Tung et al (2019), where the suggested approach performs simultaneous fixed-effect selection and parameter estimation via variational Bayes and Bayesian adaptive lasso. However, the approach is limited to high-dimensional two-level generalized linear mixed models and does not account for any streamlined updating improvements.…”

Section: Contribution and Article Organizationmentioning

confidence: 99%

Sparse Linear Mixed Model Selection via Streamlined Variational Bayes

Degani¹,

Maestrini²,

Toczydlowska³

et al. 2021

Preprint

View full text Add to dashboard Cite

Linear mixed models are a versatile statistical tool to study data by accounting for fixed effects and random effects from multiple sources of variability. In many situations, a large number of candidate fixed effects is available and it is of interest to select a parsimonious subset of those being effectively relevant for predicting the response variable. Variational approximations facilitate fast approximate Bayesian inference for the parameters of a variety of statistical models, including linear mixed models. However, for models having a high number of fixed or random effects, simple application of standard variational inference principles does not lead to fast approximate inference algorithms, due to the size of model design matrices and inefficient treatment of sparse matrix problems arising from the required approximating density parameters updates. We illustrate how recently developed streamlined variational inference procedures can be generalized to make fast and accurate inference for the parameters of linear mixed models with nested random effects and global-local priors for Bayesian fixed effects selection. Our variational inference algorithms achieve convergence to the same optima of their standard implementations, although with significantly lower computational effort, memory usage and time, especially for large numbers of random effects. Using simulated and real data examples, we assess the quality of automated procedures for fixed effects selection that are free from hyperparameters tuning and only rely upon variational posterior approximations. Moreover, we show high accuracy of variational approximations against model fitting via Markov Chain Monte Carlo sampling.

show abstract

Bayesian adaptive lasso with variational Bayes for variable selection in high-dimensional generalized linear mixed models

Cited by 7 publications

References 9 publications

Sparse linear mixed model selection via streamlined variational Bayes

Sparse linear mixed model selection via streamlined variational Bayes

A Variational Inference Framework for Inverse Problems

Sparse Linear Mixed Model Selection via Streamlined Variational Bayes

Contact Info

Product

Resources

About