Spike-and-Slab Group Lasso for Consistent Estimation and Variable Selection in Non-Gaussian Generalized Additive Models

Bai, Ray

doi:10.48550/arxiv.2007.07021

Cited by 2 publications

(14 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…= * . This converges to the SB-GAM 14 . Conversely, it is also possible to relax the assumption such that each coefficient ∈ * has its own latent indicator , but at the cost of complicating the bi-level functional selection.…”

Section: Two-part Spike-and-slab Lasso Priormentioning

confidence: 80%

“…The SSL also mitigates the problem of EMVS where the weak signals are not shrink to zero, and hence is preferred in high-dimensional data analysis. We notice that Bai 14 is the first to apply spike-and-slab lasso prior in the GAM framework, where the densities of the spike and slab components take the group lasso density 11 and limits to an "all-in-all-out" strategy for functional selection.…”

Section: Spike-and-slab Spline Priorsmentioning

confidence: 99%

“…Moreover, the EM-IWLS is a more general model fitting algorithm that can be used for fitting not only SSL and continuous SS priors but also Student's t-priors and double exponential priors. To note, SB-GAM 13,14 also used an EM-CD algorithm. The main difference between proposed EM-CD algorithm and that in SB-GAM is that SB-GAM uses a block CD algorithm for their group prior, while the proposed prior is pairwise independent requiring no special treatment in the CD algorithm.…”

Section: Algorithms For Fitting Bhamsmentioning

confidence: 99%

“…In this section, we compare the performance of the proposed models to four alternative models: component selection and smoothing operator (COSSO) 37 , adaptive COSSO 38 , generalized additive models with automatic smoothing 39 , SB-GAM 14 . COSSO is one of the earliest smoothing spline models that consider sparsity-smoothness penalty.…”

Section: Simulation Studymentioning

confidence: 99%

“…Various group spike-and-slab priors combining with computationally intensive Markov chain Monte Carlo (MCMC) algorithms 11,12 are proposed, where the application on AMs are by-products. Bai et al 13 was the first to apply group spike-andslab lasso prior to Gaussian AMs using a fast optimization algorithm, and further generalized the framework to GAMs 14 . Focus on addressing the sparsity, these methods can overly penalize the basis function coefficients and produce inaccurate predictions and curve interpolation, particularly when complex signals are assumed and large number of knots are used.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis

Guo,

Jaeger,

Rahman

et al. 2021

Preprint

View full text Add to dashboard Cite

There are proposals that extend the classical generalized additive models (GAMs) to accommodate high-dimensional data ( >> ) using group sparse regularization. However, the sparse reuglarization may induce excess shrinkage when estimating smoothing functions, damaging predictive performance. Moreover, most of these GAMs consider an "all-in-all-out" approach for functional selection, rendering them difficult to answer if nonlinear effects are necessary. While some Bayesian models can address these shortcomings, using Markov chain Monte Carlo algorithms for model fitting creates a new challenge, scalability. Hence, we propose Bayesian hierarchical generalized additive models as a solution: we consider the smoothing penalty for proper shrinkage of curve interpolation and separation of smoothing function linear and nonlinear spaces. A novel spike-and-slab spline prior is proposed to select components of smoothing functions. Two scalable and deterministic algorithms, EM-Coordinate Descent and EM-Iterative Weighted Least Squares, are developed for different utilities. Simulation studies and metabolomics data analyses demonstrate improved predictive or computational performance against state-of-theart models, mgcv, COSSO and sparse Bayesian GAM. The software implementation of the proposed models is freely available via an R package BHAM.

show abstract

Section: Two-part Spike-and-slab Lasso Priormentioning

confidence: 80%

Section: Spike-and-slab Spline Priorsmentioning

confidence: 99%

Section: Algorithms For Fitting Bhamsmentioning

confidence: 99%

Section: Simulation Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis

Guo,

Jaeger,

Rahman

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Spike‐and‐slab least absolute shrinkage and selection operator generalized additive models and scalable algorithms for high‐dimensional data analysis

Guo

Jaeger

Rahman

et al. 2022

Statistics in Medicine

View full text Add to dashboard Cite

There are proposals that extend the classical generalized additive models (GAMs) to accommodate high-dimensional data (p ≫ n) using group sparse regularization. However, the sparse regularization may induce excess shrinkage when estimating smooth functions, damaging predictive performance. Moreover, most of these GAMs consider an "all-in-all-out" approach for functional selection, rendering them difficult to answer if nonlinear effects are necessary.While some Bayesian models can address these shortcomings, using Markov chain Monte Carlo algorithms for model fitting creates a new challenge, scalability. Hence, we propose Bayesian hierarchical generalized additive models as a solution: we consider the smoothing penalty for proper shrinkage of curve interpolation via reparameterization. A novel two-part spike-and-slab LASSO prior for smooth functions is developed to address the sparsity of signals while providing extra flexibility to select the linear or nonlinear components of smooth functions. A scalable and deterministic algorithm, EM-Coordinate Descent, is implemented in an open-source R package BHAM. Simulation studies and metabolomics data analyses demonstrate improved predictive and computational performance against state-of-the-art models. Functional selection performance suggests trade-offs exist regarding the effect hierarchy assumption.

show abstract

Spike-and-Slab Group Lasso for Consistent Estimation and Variable Selection in Non-Gaussian Generalized Additive Models

Cited by 2 publications

References 24 publications

Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis

Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis

Spike‐and‐slab least absolute shrinkage and selection operator generalized additive models and scalable algorithms for high‐dimensional data analysis

Contact Info

Product

Resources

About