Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models

Bai, Ray; Moran, Gemma E.; Antonelli, Joseph; Chen, Yong; Boland, Mary Regina

doi:10.1080/01621459.2020.1765784

Cited by 35 publications

(24 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This prior can be reformulated as a likelihood penalty function that represents a combination of weak penalization of larger effects by λ 1, l , k and strong penalization of effects close to zero by λ 0, l , k , respectively (See Supplementary Material Section 1.2 ). As recommended by Ročková and George (2018) , we use the non-separable version of the spike-and-slab lasso prior, which provides self-adaptivity of the sparsity level and an automatic control for multiplicity via a Beta prior on θ ( Bai et al (2020a) ; Scott and Berger (2010) ). We further set λ 0, l , k = 50 ∀l , k to achieve a strong penalization in the “spike” part of the prior, leaving λ 1, l , k as our only parameter that controls the total amount of penalty applied at larger effect values.…”

Section: Methodsmentioning

confidence: 99%

tascCODA: Bayesian Tree-Aggregated Analysis of Compositional Amplicon and Single-Cell Data

2021

View full text Add to dashboard Cite

Accurate generative statistical modeling of count data is of critical relevance for the analysis of biological datasets from high-throughput sequencing technologies. Important instances include the modeling of microbiome compositions from amplicon sequencing surveys and the analysis of cell type compositions derived from single-cell RNA sequencing. Microbial and cell type abundance data share remarkably similar statistical features, including their inherent compositionality and a natural hierarchical ordering of the individual components from taxonomic or cell lineage tree information, respectively. To this end, we introduce a Bayesian model for tree-aggregated amplicon and single-cell compositional data analysis (tascCODA) that seamlessly integrates hierarchical information and experimental covariate data into the generative modeling of compositional count data. By combining latent parameters based on the tree structure with spike-and-slab Lasso penalization, tascCODA can determine covariate effects across different levels of the population hierarchy in a data-driven parsimonious way. In the context of differential abundance testing, we validate tascCODA’s excellent performance on a comprehensive set of synthetic benchmark scenarios. Our analyses on human single-cell RNA-seq data from ulcerative colitis patients and amplicon data from patients with irritable bowel syndrome, respectively, identified aggregated cell type and taxon compositional changes that were more predictive and parsimonious than those proposed by other schemes. We posit that tascCODA1 constitutes a valuable addition to the growing statistical toolbox for generative modeling and analysis of compositional changes in microbial or cell population data.

show abstract

Section: Methodsmentioning

confidence: 99%

tascCODA: Bayesian Tree-Aggregated Analysis of Compositional Amplicon and Single-Cell Data

2021

View full text Add to dashboard Cite

show abstract

“…We use three minibatch sizes of m ∈ { n 4 , n 2 , n} to test how this affects performance. We also implement the following models with embedded variable selection: (LASSO (Tibshirani, 1996), MCP (Zhang, 2010), GAM with group LASSO penalty (Huang et al, 2009) and GAM with group spike and slab LASSO (SS LASSO) penalty (Bai et al, 2020). The LASSO and MCP were implemented using the R ncvreg package (Breheny and Breheny, 2021), the GAM LASSO and GAM spike and slab LASSO (SSwere implemented using the R package sparseGAM (Bai, 2021).…”

Section: Methodsmentioning

confidence: 99%

“…We also implement the ML-II GP and several benchmark variable selection algorithms: LASSO (Tibshirani, 1996), MCP (Zhang, 2010), GAM with group LASSO penalty (Huang et al, 2009) and GAM with group spike and slab LASSO penalty (Bai et al, 2020). We implement our SSVGP with minibatch sizes m ∈ { n 4 , n 2 , n} and the same settings as previously.…”

Section: Experiments 1: a Small-scale High Dimensional Variable Selec...mentioning

confidence: 99%

“…Bold indicates best performing method per column. All SSVGP implementations outperformed the comparator set on average variable selection and predictive accuracy, with the exception that the GAM spike and slab lasso of Bai et al (2020) marginally outperfomed the SSVGP with m = n on average with respect to variable selection accuracy (MCC). However this is mostly due to several outlier trials with poor performance for the SSVGP(m=n), which skews the average MSE and MCC.…”

Section: Msementioning

confidence: 99%

See 1 more Smart Citation

Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes

Dance¹,

Paige²

2021

Preprint

View full text Add to dashboard Cite

Variable selection in Gaussian processes (GPs) is typically undertaken by thresholding the inverse lengthscales of 'automatic relevance determination' kernels, but in highdimensional datasets this approach can be unreliable. A more probabilistically principled alternative is to use spike and slab priors and infer a posterior probability of variable inclusion. However, existing implementations in GPs are extremely costly to run in both high-dimensional and large-n datasets, or are intractable for most kernels. As such, we develop a fast and scalable variational inference algorithm for the spike and slab GP that is tractable with arbitrary differentiable kernels. We improve our algorithm's ability to adapt to the sparsity of relevant variables by Bayesian model averaging over hyperparameters, and achieve substantial speed ups using zero temperature posterior restrictions, dropout pruning and nearest neighbour minibatching. In experiments our method consistently outperforms vanilla and sparse variational GPs whilst retaining similar runtimes (even when n = 10 6 ) and performs competitively with a spike and slab GP using MCMC but runs up to 1000 times faster.

show abstract

“…This prior can be reformulated as a likelihood penalty function that finds a balance between weak and strong penalization by λ 1 and λ 0 , respectively (See Supplementary material section 1.2). As recommended by Ročková and George (2018), we use the non-separable version of the spike-and-slab lasso prior, which provides self-adaptivity of the sparsity level and an automatic control for multiplicity via a Beta prior on θ (Bai et al (2020a); Scott and Berger (2010)). We further set λ 0,l,k = 50 ∀k to achieve a strong penalization in the "spike" part of the prior, leaving λ 1,l,k as our only parameter that controls the total amount of penalty applied at larger effect values.…”

Section: Spike-and-slab Lasso Priormentioning

confidence: 99%

tascCODA: Bayesian tree-aggregated analysis of compositional amplicon and single-cell data

Ostner

Carcy

Müller

2021

Preprint

View full text Add to dashboard Cite

Accurate generative statistical modeling of count data is of critical relevance for the analysis of biological datasets from high-throughput sequencing technologies. Important instances include the modeling of microbiome compositions from amplicon sequencing surveys and the analysis of cell type compositions derived from single-cell RNA sequencing. Microbial and cell type abundance data share remarkably similar statistical features, including their inherent compositionality and a natural hierarchical ordering of the individual components from taxonomic or cell lineage tree information, respectively. To this end, we introduce a Bayesian model for tree-aggregated amplicon and single-cell compositional data analysis tascCODA that seamlessly integrates hierarchical information and experimental covariate data into the generative modeling of compositional count data. By combining latent parameters based on the tree structure with spike-and-slab Lasso penalization, tascCODA can determine covariate effects across different levels of the population hierarchy in a data-driven parsimonious way. In the context of differential abundance testing, we validate tascCODA's excellent performance on a comprehensive set of synthetic benchmark scenarios. Our analyses on human single-cell RNA-seq data from ulcerative colitis patients and amplicon data from patients with irritable bowel syndrome, respectively, identified aggregated cell type and taxon compositional changes that were more predictive and parsimonious than those proposed by other schemes. We posit that tascCODA constitutes a valuable addition to the growing statistical toolbox for generative modeling and analysis of compositional changes in microbial or cell population data.

show abstract

Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models

Cited by 35 publications

References 46 publications

tascCODA: Bayesian Tree-Aggregated Analysis of Compositional Amplicon and Single-Cell Data

tascCODA: Bayesian Tree-Aggregated Analysis of Compositional Amplicon and Single-Cell Data

Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes

tascCODA: Bayesian tree-aggregated analysis of compositional amplicon and single-cell data

Contact Info

Product

Resources

About