iDNA-ABT: advanced deep learning model for detecting DNA methylation with adaptive features and transductive information maximization

Abstract

Motivation DNA methylation plays an important role in epigenetic modification, the occurrence, and the development of diseases. Therefore, the identification of DNA methylation sites is critical for better understanding and revealing their functional mechanisms. To date, several machine learning and deep learning methods have been developed for the prediction of different methylation types. However, they still highly rely on manual features, which can largely limit the high-latent information extraction. Moreover, most of them are designed for one specific methylation type, and therefore cannot predict multiple methylation sites in multiple species simultaneously. In this study, we propose iDNA-ABT, an advanced deep learning model that utilizes adaptive embedding based on bidirectional transformers for language understanding together with a novel transductive information maximization (TIM) loss. Results Benchmark results show that our proposed iDNA-ABT can automatically and adaptively learn the distinguishing features of biological sequences from multiple species, and thus perform significantly better than the state-of-the-art methods in predicting three different DNA methylation. In addition, TIM loss is proven to be effective in dichotomous tasks via the comparison experiment. Furthermore, we verify that our features have strong adaptability and robustness to different species through comparison of adaptive embedding and six handcrafted feature encodings. Importantly, our model shows great generalization ability in different species, demonstrating that our model can adaptively capture the cross-species differences and improve the predictive performance. For the convenient use of our method, we further established an online webserver as the implementation of the proposed iDNA-ABT. Availability our proposed iDNA-ABT, which is now freely accessible via http://server.wei-group.net/iDNA_ABT and our source codes are available in the GitHub repository (https://github.com/YUYING07/iDNA_ABT) Supplementary information Supplementary data are available at Bioinformatics online.

MuLan-Methyl - Multiple Transformer-based Language Models for Accurate DNA Methylation Prediction

“…There is a large number of papers that address the problem of identifying methylation sites, however, most of them focus on specific form of modification (10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29), and only a few methods address all three types of methylation mentioned above (30)(31)(32)(33)(34), including iDNA-MS, iDNA-ABT, and iDNA-ABF. Note that the database presented in (31) is now widely used as a benchmark dataset for assessing model performance (21,23,(32)(33)(34).…”

“…Here, we phrase DNA methylation-site detection as a Natural Language Processing (NLP) problem and propose a novel framework to address it. Previous studies for identifying methylation sites usually use BERT, a classic NLP approach, or, in the context of DNA sequences, the variant DNABERT (36), either as a model that accepts embeddings from Word2vec, or as an encoder that generates embeddings for input to a deep neural network (23, 25, 32, 33, 37).…”

CNN6mA: Interpretable neural network model based on position-specific CNN and cross-interactive network for 6mA site prediction

“…Huang et al combined LSTM with attention mechanisms and presented comparable performances to SICD6mA in 6 mA site prediction [23] . Yu et al constructed the BERT-based neural network model, named iDNA-ABT [24] , and compared it with the previous models including iDNA-MS and SNNRice6mA on the benchmark datasets constructed by Lv et al (iDNA-MS). The BERT-based neural network generated the feature vectors, depending on the context information to extract different information exhaustively.…”

Improving DNA 6mA Site Prediction via Integrating Bidirectional Long Short-Term Memory, Convolutional Neural Network, and Self-Attention Mechanism

“…The method in ref employed bidirectional encoder representations from transformers (BERT), which is a new language representation model, to dig out the dependency information from both left and right contexts for each nucleotide. iDNA-ABT proposed an advanced deep learning model that utilizes adaptive embedding based on BERT together with transductive information maximization. TC-6mA-Pred employed the deep transformer neural network to accurately predict the 6mA sites.…”