4mCPred-MTL: Accurate Identification of DNA 4mC Sites in Multiple Species Using Multi-Task Deep Learning Based on Multi-Head Attention Mechanism

Zeng, Rao; Cheng, Song; Liao, Minghong

doi:10.3389/fcell.2021.664669

Cited by 10 publications

(6 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the RNN or CNN-RNN models, relatively important features can be effectively captured by introducing a self-attention mechanism. This remarkably complex mechanism has been widely used for classification tasks in various fields ( Hu et al, 2019 ; Liang et al, 2021 ; Tian et al, 2021 ), including the prediction of 4mC sites ( Zeng and Liao, 2020 ; Liu et al, 2021 ; Xu et al, 2021 ; Zeng et al, 2021 ). To test whether the attention mechanism improves the predictive performance, we implemented the self-attention layer in different network architectures of RNN and CNN-RNN.…”

Section: Resultsmentioning

confidence: 99%

Systematic Analysis and Accurate Identification of DNA N4-Methylcytosine Sites by Deep Learning

Zhang

et al. 2022

Front. Microbiol.

View full text Add to dashboard Cite

DNA N4-methylcytosine (4mC) is a pivotal epigenetic modification that plays an essential role in DNA replication, repair, expression and differentiation. To gain insight into the biological functions of 4mC, it is critical to identify their modification sites in the genomics. Recently, deep learning has become increasingly popular in recent years and frequently employed for the 4mC site identification. However, a systematic analysis of how to build predictive models using deep learning techniques is still lacking. In this work, we first summarized all existing deep learning-based predictors and systematically analyzed their models, features and datasets, etc. Then, using a typical standard dataset with three species (A. thaliana, C. elegans, and D. melanogaster), we assessed the contribution of different model architectures, encoding methods and the attention mechanism in establishing a deep learning-based model for the 4mC site prediction. After a series of optimizations, convolutional-recurrent neural network architecture using the one-hot encoding and attention mechanism achieved the best overall prediction performance. Extensive comparison experiments were conducted based on the same dataset. This work will be helpful for researchers who would like to build the 4mC prediction models using deep learning in the future.

show abstract

Section: Resultsmentioning

confidence: 99%

Systematic Analysis and Accurate Identification of DNA N4-Methylcytosine Sites by Deep Learning

Zhang

et al. 2022

Front. Microbiol.

View full text Add to dashboard Cite

show abstract

“…There is a large number of papers that address the problem of identifying methylation sites, however, most of them focus on specific form of modification (10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29), and only a few methods address all three types of methylation mentioned above (30)(31)(32)(33)(34), including iDNA-MS, iDNA-ABT, and iDNA-ABF. Note that the database presented in (31) is now widely used as a benchmark dataset for assessing model performance (21,23,(32)(33)(34).…”

Section: Introductionmentioning

confidence: 99%

MuLan-Methyl - Multiple Transformer-based Language Models for Accurate DNA Methylation Prediction

Zeng

Gautam

Huson

2023

Preprint

View full text Add to dashboard Cite

Transformer-based language models are successfully used to address massive text-related tasks. DNA methylation is an important epigenetic mechanism and its analysis provides valuable insights into gene regulation and biomarker identification. Several deep learning-based methods have been proposed to identify DNA methylation and each seeks to strike a balance between computational effort and accuracy. Here, we introduce a MuLan-Methyl, a deep learning framework for predicting DNA methylation sites, which is based on multiple (five) popular transformer-based language models. The framework identifies methylation sites for three different types of DNA methylation, namely N6-adenine (6mA), N4-cytosine (4mC), and 5-hydroxymethylcytosine (5hmC). Each of the five employed language models is adapted to the task using the "pre-train and fine-tune" paradigm. Pre-training is performed on a custom corpus consisting of DNA fragments and taxonomy lineages using self-supervised learning. Fine-tuning then aims at predicting the DNA-methylation status of each type. The five models are used to collectively predict the DNA methylation status. We report excellent performance of MuLan-Methyl on a benchmark dataset. Moreover, we show that the model captures characteristic differences between different species that are relevant for methylation. This work demonstrates that language models can be successfully adapted to this domain of application and that joint utilization of different language models improves model performance.

show abstract

“…Most recently, Tang et al developed DNA4mC-LIP [29] that linearly integrated all the aforementioned methods for the prediction of 4mC sites. Besides models developed using classic machine learning algorithms, multiple deep learning models were developed to identify 4mC sites [30] , [31] , [32] , [33] , [34] , [35] . Despite the fact that these approaches yielded satisfactory outcomes, there is still much room for improvement.…”

Section: Introductionmentioning

confidence: 99%

i4mC-GRU: Identifying DNA N4-Methylcytosine sites in mouse genomes using bidirectional gated recurrent unit and sequence-embedded features

Nguyen-Vo

Trinh

Nguyen

et al. 2023

Computational and Structural Biotechnology Journal

View full text Add to dashboard Cite

4mCPred-MTL: Accurate Identification of DNA 4mC Sites in Multiple Species Using Multi-Task Deep Learning Based on Multi-Head Attention Mechanism

Cited by 10 publications

References 89 publications

Systematic Analysis and Accurate Identification of DNA N4-Methylcytosine Sites by Deep Learning

Systematic Analysis and Accurate Identification of DNA N4-Methylcytosine Sites by Deep Learning

MuLan-Methyl - Multiple Transformer-based Language Models for Accurate DNA Methylation Prediction

i4mC-GRU: Identifying DNA N4-Methylcytosine sites in mouse genomes using bidirectional gated recurrent unit and sequence-embedded features

Contact Info

Product

Resources

About