Learning Cell-Type-Specific Gene Regulation Mechanisms by Multi-Attention Based Deep Learning With Regulatory Latent Space

Kang, Moon-Jin; Lee, Sangseon; Lee, Jun Young; Kim, Sun

doi:10.3389/fgene.2020.00869

Cited by 13 publications

(14 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kang et al . [ 73 ] used an attention mechanism for multi-omics data [ 99 ] for interpreting predictions of gene expression. Generally, these gradient- and attention-based methods show that heterogeneous intermediate fusion does not impede models that allow sound biological interpretation.…”

Section: Intermediate Fusionmentioning

confidence: 99%

Multimodal deep learning for biomedical data fusion: a review

Stahlschmidt

Ulfenborg

Synnergren

2022

Briefings in Bioinformatics

211

View full text Add to dashboard Cite

Biomedical data are becoming increasingly multimodal and thereby capture the underlying complex relationships among biological processes. Deep learning (DL)-based data fusion strategies are a popular approach for modeling these nonlinear relationships. Therefore, we review the current state-of-the-art of such methods and propose a detailed taxonomy that facilitates more informed choices of fusion strategies for biomedical applications, as well as research on novel methods. By doing so, we find that deep fusion strategies often outperform unimodal and shallow approaches. Additionally, the proposed subcategories of fusion strategies show different advantages and drawbacks. The review of current methods has shown that, especially for intermediate fusion strategies, joint representation learning is the preferred approach as it effectively models the complex interactions of different levels of biological organization. Finally, we note that gradual fusion, based on prior biological knowledge or on search strategies, is a promising future research path. Similarly, utilizing transfer learning might overcome sample size limitations of multimodal data sets. As these data sets become increasingly available, multimodal DL approaches present the opportunity to train holistic models that can learn the complex regulatory dynamics behind health and disease.

show abstract

Section: Intermediate Fusionmentioning

confidence: 99%

Multimodal deep learning for biomedical data fusion: a review

Stahlschmidt

Ulfenborg

Synnergren

2022

Briefings in Bioinformatics

211

View full text Add to dashboard Cite

show abstract

“…However, there are other regulatory mechanisms, such as mutations, copy number variations, and epigenetic mechanisms, that can affect transcription level of genes. This requires a comprehensive model, e.g., ensemble of deep learning (Kang et al, 2020 ). Combining network analysis techniques and deep learning technologies is a major current research topic.…”

Section: Resultsmentioning

confidence: 99%

Construction of Condition-Specific Gene Regulatory Network Using Kernel Canonical Correlation Analysis

et al. 2021

Self Cite

View full text Add to dashboard Cite

Gene expression profile or transcriptome can represent cellular states, thus understanding gene regulation mechanisms can help understand how cells respond to external stress. Interaction between transcription factor (TF) and target gene (TG) is one of the representative regulatory mechanisms in cells. In this paper, we present a novel computational method to construct condition-specific transcriptional networks from transcriptome data. Regulatory interaction between TFs and TGs is very complex, specifically multiple-to-multiple relations. Experimental data from TF Chromatin Immunoprecipitation sequencing is useful but produces one-to-multiple relations between TF and TGs. On the other hand, co-expression networks of genes can be useful for constructing condition transcriptional networks, but there are many false positive relations in co-expression networks. In this paper, we propose a novel method to construct a condition-specific and combinatorial transcriptional network, applying kernel canonical correlation analysis (kernel CCA) to identify multiple-to-multiple TF–TG relations in certain biological condition. Kernel CCA is a well-established statistical method for computing the correlation of a group of features vs. another group of features. We, therefore, employed kernel CCA to embed TFs and TGs into a new space where the correlation of TFs and TGs are reflected. To demonstrate the usefulness of our network construction method, we used the blood transcriptome data for the investigation on the response to high fat diet in a human and an arabidopsis data set for the investigation on the response to cold/heat stress. Our method detected not only important regulatory interactions reported in previous studies but also novel TF–TG relations where a module of TF is regulating a module of TGs upon specific stress.

show abstract

“…We also trained the AttentiveChrome 3 models that combines LSTM and global attention mechanism to enhance the interpretability of the model. Lastly, an HM-based hybrid CNN-RNN model proposed by Kang et al 18 (HM-CRNN) was also chosen for the comparison. It learns and captures the meaningful local combination of HMs through CNN and comprehends their sequential dependencies through RNN.…”

Section: Chromoformer Outperforms Existing Deep Models For Gene Expression Prediction Based On Epigenetic Featuresmentioning

confidence: 99%

Learning the histone codes of gene regulation with large genomic windows and three-dimensional chromatin interactions using transformer

Lee

Yang

Kim

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The quantitative characterization of the transcriptional control by histone modifications (HMs) has been challenged by many computational studies, but still most of them exploit only partial aspects of intricate mechanisms involved in gene regulation, leaving a room for improvement. We present Chromoformer, a new transformer-based deep learning architecture that achieves the state-of-the-art performance in the quantitative deciphering of the histone codes of gene regulation. The core essence of Chromoformer architecture lies in the three variants of attention operation, each specialized to model individual hierarchy of three-dimensional (3D) transcriptional regulation including (1) histone codes at core promoters, (2) pairwise interaction between a core promoter and a distal cis-regulatory element mediated by 3D chromatin interactions, and (3) the collective effect of the pairwise cis-regulations. In-depth interpretation of the trained model behavior based on attention scores suggests that Chromoformer adaptively exploits the distant dependencies between HMs associated with transcription initiation and elongation. We also demonstrate that the quantitative kinetics of transcription factories and polycomb group bodies, in which the coordinated gene regulation occurs through spatial sequestration of genes with regulatory elements, can be captured by Chromoformer. Together, our study shows the great power of attention-based deep learning as a versatile modeling approach for the complex epigenetic landscape of gene regulation and highlights its potential as an effective toolkit that facilitates scientific discoveries in computational epigenetics.

show abstract

Learning Cell-Type-Specific Gene Regulation Mechanisms by Multi-Attention Based Deep Learning With Regulatory Latent Space

Cited by 13 publications

References 36 publications

Multimodal deep learning for biomedical data fusion: a review

Multimodal deep learning for biomedical data fusion: a review

Construction of Condition-Specific Gene Regulatory Network Using Kernel Canonical Correlation Analysis

Learning the histone codes of gene regulation with large genomic windows and three-dimensional chromatin interactions using transformer

Contact Info

Product

Resources

About