Transfer learning enables predictions in network biology

Theodoris, Christina V.; Xiao, Ling; Chopra, Anant; Chaffin, Mark D.; Al Sayed, Zeina R.; Hill, Matthew C.; Mantineo, Helene; Brydon, Elizabeth M.; Zeng, Zexian; Liu, X. Shirley; Ellinor, Patrick T.

doi:10.1038/s41586-023-06139-9

Cited by 92 publications

(16 citation statements)

References 155 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Seq2cells rests on the premise that, using pretrained epigenome models to create DNA sequence embeddings of the TSS, we can create gene embeddings that encapsulate transcriptional regulation. This allows us to predict gene expression from the sequence determinants, for example TF motifs, that drive gene expression mechanistically, rather than from co-expression patterns of genes [39][40][41]. In the future, we hope to aid other gene-level task with the information contained in the regulatory DNA sequence of a gene.…”

Section: Disussionmentioning

confidence: 99%

See 1 more Smart Citation

Single-cell gene expression prediction from DNA sequence at large contexts

Deasy¹,

T.²,

Young³

et al. 2023

Preprint

View full text Add to dashboard Cite

Human genetic variants impacting traits such as disease susceptibility frequently act through modulation of gene expression in a highly cell-type-specific manner. Computational models capable of predicting gene expression directly from DNA sequence can assist in the interpretation of expression-modulating variants, and machine learning models now operate at the large sequence contexts required for capturing long-range human transcriptional regulation. However, existing predictors have focused on bulk transcriptional measurements where gene expression heterogeneity can be drowned out in broadly defined cell types. Here, we use a transfer learning framework, seq2cells, leveraging a pre-trained epigenome model for gene expression prediction from large sequence contexts at single-cell resolution. We show that seq2cells captures cell-specific gene expression beyond the resolution of pseudo-bulked data. Using seq2cells for variant effect prediction reveals heterogeneity within annotated cell types and enables in silico transfer of variant effects between cell populations. We demonstrate the challenges and value of gene expression and variant effect prediction at single-cell resolution, and offer a path to the interpretation of genomic variation at uncompromising resolution and scale.

show abstract

Section: Disussionmentioning

confidence: 99%

“…Here we successfully trained a single-cell expression model on 250 k cells and with compromising on the bottleneck dimension on 650 k cells. Purely transcriptomics-based single-cell models [39][40][41] demonstrated the benefits of scaling to millions of cells. We envisage similar benefits from scaling sequence-based models into this realm.…”

Section: Disussionmentioning

confidence: 99%

Single-cell gene expression prediction from DNA sequence at large contexts

Deasy¹,

T.²,

Young³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…We evaluated two proposed foundation models for single-cell transcriptomics: Geneformer [5] and scGPT [6]. We chose these models because they offer pretrained weights (whereas several other possible models did not have public weights at time of evaluation) and have been trained using unsupervised objectives on extensive datasets (ca.…”

Section: Models and Baselinesmentioning

confidence: 99%

Assessing the limits of zero-shot foundation models in single-cell biology

Kedzierska,

Crawford,

Amini

et al. 2023

Preprint

View full text Add to dashboard Cite

The advent and success of foundation models such as GPT has sparked growing interest in their application to single-cell biology. Models like Geneformer and scGPT have emerged with the promise of serving as versatile tools for this specialized field. However, the efficacy of these models, particularly in zero-shot settings where models are not fine-tuned but used without any further training, remains an open question, especially as practical constraints require useful models to function in settings that preclude fine-tuning (e.g., discovery settings where labels are not fully known). This paper presents a rigorous evaluation of the zero-shot performance of these proposed single-cell foundation models. We assess their utility in tasks such as cell type clustering and batch effect correction, and evaluate the generality of their pretraining objectives. Our results indicate that both Geneformer and scGPT exhibit limited reliability in zero-shot settings and often underperform compared to simpler methods. These findings serve as a cautionary note for the deployment of proposed single-cell foundation models and highlight the need for more focused research to realize their potential. The code used for our analyses can be accessed at https://github.com/microsoft/zero-shot-scfoundation.

show abstract

“…Instead, they tend to be located in disease gene modules 11,12 . Dissecting the gene modules that drive disease progression enables screening for the molecules that correct the network rather than targeting peripheral downstream effectors that may not be disease modifying [13][14][15] . Active driver modules can trigger the hallmarks of cancer and confer fitness advantages to cancer cells 16,17 .…”

Section: Introductionmentioning

confidence: 99%

CGMega: Explainable Graph Neural Network Framework with Attention Mechanisms for Cancer Gene Module Dissection

Li¹,

Han

Sun

et al. 2023

Preprint

View full text Add to dashboard Cite

Cancer is rarely the straightforward consequence of an abnormality in a single gene, but rather reflects a complex interplay of many genes, represented as gene modules. Here, we leveraged the recent advances of model-agnostic interpretation approach and developed CGMega, an explainable and graph attention-based deep learning framework to perform cancer gene module dissection. CGMega outperforms current approaches in cancer gene prediction, and it provides a promising approach to integrate multi-omics information. We applied CGMega to breast cancer cell line and acute myeloid leukemia (AML) patients, and we uncovered the high-order gene module formed by ErbB family and tumor factors NRG1, PPM1A and DLG2. We identified 396 candidate AML genes, and observed the enrichment of either known AML genes or candidate AML genes in a single gene module. We also identified patient-specific AML genes and associated gene modules. Together, these results indicate that CGMega can be used to dissect cancer gene modules, and provide high-order mechanistic insights into cancer development and heterogeneity.

show abstract

Transfer learning enables predictions in network biology

Cited by 92 publications

References 155 publications

Single-cell gene expression prediction from DNA sequence at large contexts

Single-cell gene expression prediction from DNA sequence at large contexts

Assessing the limits of zero-shot foundation models in single-cell biology

CGMega: Explainable Graph Neural Network Framework with Attention Mechanisms for Cancer Gene Module Dissection

Contact Info

Product

Resources

About