Rare Feature Selection in High Dimensions

Yan, Xiaohan; Bien, Jacob

doi:10.1080/01621459.2020.1796677

Cited by 35 publications

(57 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The choice of the ℓ 1 penalty was motivated in [15] by the high dimensionality of microbiome data and the desire for parsimonious predictive models. However, such a penalty is not well-suited to situations in which large numbers of features are highly rare [21], a well-known feature of amplicon data. A common remedy, also adopted in [15], is to aggregate taxa at the base level, e.g., OTUs or ASVs, to the genus level and then to screen out all but the most abundant genera.…”

Section: Methodsmentioning

confidence: 99%

“…Using OTU/ASVs as base level , Figure 1A illustrates the typical aggregation-to-genus level approach whereas Figure 1B shows the prediction-dependent approach. The method is designed to mesh seamlessly with the compositional data analysis framework by combining log-contrast regression [20] with tree-guided regularization, recently put forward in [21]. Thanks to the convexity of the underlying penalized estimation problem, can deliver interpretable aggregated solutions to large-scale microbiome regression problems in a fast and reproducible manner.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Tree-Aggregated Predictive Modeling of Microbiome Data

Bien

Yan

Simpson

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Modern high-throughput sequencing technologies provide low-cost microbiome survey data across all habitats of life at unprecedented scale. At the most granular level, the primary data consist of sparse counts of amplicon sequence variants or operational taxonomic units that are associated with taxonomic and phylogenetic group information. In this contribution, we leverage the hierarchical structure of amplicon data and propose a data-driven, parameter-free, and scalable tree-guided aggregation framework to associate microbial subcompositions with response variables of interest. The excess number of zero or low count measurements at the read level forces traditional microbiome data analysis workflows to remove rare sequencing variants or group them by a fixed taxonomic rank, such as genus or phylum, or by phylogenetic similarity. By contrast, our framework, which we call trac (tree-aggregation of compositional data), learns data-adaptive taxon aggregation levels for predictive modeling making user-defined aggregation obsolete while simultaneously integrating seamlessly into the compositional data analysis framework. We illustrate the versatility of our framework in the context of large-scale regression problems in human-gut, soil, and marine microbial ecosystems. We posit that the inferred aggregation levels provide highly interpretable taxon groupings that can help microbial ecologists gain insights into the structure and functioning of the underlying ecosystem of interest.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Tree-Aggregated Predictive Modeling of Microbiome Data

Bien

Yan

Simpson

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Yan and Bien 20 are independently developing a similar method involving adding aggregated variables to alter the regularization. Their approach is tailored to text mining problems, and consequently differs from ours in a couple of respects: First, they include an additional penalty for the coefficients at leaf nodes.…”

Section: Adapting Surf To Tree Structured Datamentioning

confidence: 99%

SuRF: A new method for sparse variable selection, with application in microbiome data analysis

Liu

Limbergen

et al. 2020

Statistics in Medicine

View full text Add to dashboard Cite

In this article, we present a new variable selection method for regression and classification purposes, particularly for microbiome analysis. Our method, called subsampling ranking forward selection (SuRF), is based on LASSO penalized regression, subsampling and forward‐selection methods. SuRF offers major advantages over existing variable selection methods in terms of both sparsity of selected models and model inference. We provide an R package that can implement our method for generalized linear models. We apply our method to classification problems from microbiome data, using a novel agglomeration approach to deal with the special tree‐like correlation structure of the variables. Existing methods arbitrarily choose a taxonomic level a priori before performing the analysis, whereas by combining SuRF with these aggregated variables, we are able to identify the key biomarkers at the appropriate taxonomic level, as suggested by the data. We present simulations in multiple sparse settings to demonstrate that our approach performs better than several other popularly used existing approaches in recovering the true variables. We apply SuRF to two microbiome datasets: one about prediction of pouchitis and another for identifying samples from two healthy individuals. We find that SuRF can provide a better or comparable prediction with other methods while controlling the false positive rate of variable selection.

show abstract

“…In recent years, several tree-guided lasso methods have been developed. For example, TASSO [ 20 ] applies an l 1 penalty on the sum of coefficients within each possible subtree, while rare [ 21 ] applies an l 1 penalty on latent variables of nodes to induce subtrees having equal coefficient values. Citrus [ 22 ] works on CyTOF data and applies a lasso-regularized regression model [ 23 ] to automatically select stratifying subpopulations and cell response features that are the best predictors of a phenotypic outcome.…”

Section: Introductionmentioning

confidence: 99%

treeclimbR pinpoints the data-dependent resolution of hierarchical hypotheses

et al. 2021

View full text Add to dashboard Cite

AbstracttreeclimbR is for analyzing hierarchical trees of entities, such as phylogenies or cell types, at different resolutions. It proposes multiple candidates that capture the latent signal and pinpoints branches or leaves that contain features of interest, in a data-driven way. It outperforms currently available methods on synthetic data, and we highlight the approach on various applications, including microbiome and microRNA surveys as well as single-cell cytometry and RNA-seq datasets. With the emergence of various multi-resolution genomic datasets, treeclimbR provides a thorough inspection on entities across resolutions and gives additional flexibility to uncover biological associations.

show abstract

Rare Feature Selection in High Dimensions

Cited by 35 publications

References 37 publications

Tree-Aggregated Predictive Modeling of Microbiome Data

Tree-Aggregated Predictive Modeling of Microbiome Data

SuRF: A new method for sparse variable selection, with application in microbiome data analysis

treeclimbR pinpoints the data-dependent resolution of hierarchical hypotheses

Contact Info

Product

Resources

About