Leveraging the attention mechanism to improve the identification of DNA N6-methyladenine sites

Abstract

DNA N6-methyladenine is an important type of DNA modification that plays important roles in multiple biological processes. Despite the recent progress in developing DNA 6mA site prediction methods, several challenges remain to be addressed. For example, although the hand-crafted features are interpretable, they contain redundant information that may bias the model training and have a negative impact on the trained model. Furthermore, although deep learning (DL)-based models can perform feature extraction and classification automatically, they lack the interpretability of the crucial features learned by those models. As such, considerable research efforts have been focused on achieving the trade-off between the interpretability and straightforwardness of DL neural networks. In this study, we develop two new DL-based models for improving the prediction of N6-methyladenine sites, termed LA6mA and AL6mA, which use bidirectional long short-term memory to respectively capture the long-range information and self-attention mechanism to extract the key position information from DNA sequences. The performance of the two proposed methods is benchmarked and evaluated on the two model organisms Arabidopsis thaliana and Drosophila melanogaster. On the two benchmark datasets, LA6mA achieves an area under the receiver operating characteristic curve (AUROC) value of 0.962 and 0.966, whereas AL6mA achieves an AUROC value of 0.945 and 0.941, respectively. Moreover, an in-depth analysis of the attention matrix is conducted to interpret the important information, which is hidden in the sequence and relevant for 6mA site prediction. The two novel pipelines developed for DNA 6mA site prediction in this work will facilitate a better understanding of the underlying principle of DL-based DNA methylation site prediction and its future applications.

I-DNAN6mA: Accurate Identification of DNA N6-Methyladenine Sites Using the Base-Pairing Map and Deep Learning

“…The Rice genome containing both 6mA-rice-Chen and 6mA-rice-Lv data sets, which are widely used to examine the performance of the previous methods ,,, over cross-validation tests, are also employed to evaluate our method by performing cross-validation tests. For more detailed information on the data set construction, please refer to refs , and –. Table shows the details of the statistical composition of these data sets.…”

“…To overcome the drawback of ML-based methods, a few DL techniques that utilize multi-layer artificial neural networks to learn tasks have been successfully applied to solve computational biology problems, including 6mA site prediction. A few of these have been named below: SpineNet-6mA, GC6mA-Pred, iRicem6A-CNN, Deep6mA, i6mA-VC, SNNRice6mA, DeepM6A, iDNA6mA (five-step rule), i6mA-DNC, LA6mA, AL6mA, 6mA-RicePred, and Deep6mAPred . These DL-based methods often utilize only one-hot encoding of the DNA sequence to recognize 6mA sites with one or more DL algorithms, such as convolutional neural networks and fully connected hidden layers .…”

DNA N6-Methyladenine Modification in Eukaryotic Genome

“…Compared with traditional laboratory experiments, bioinformatics tools have significant advantages in terms of price and time cost (Figure 1D). At present, there are many deep learning models used for predicting 6 mA, such as DNA6mA-MINT (Rehman and Chong, 2020), i6mA-stack (Khanal et al, 2021), SNNRice6mA (Yu and Dai, 2019), SMEP (Wang et al, 2021), Deep6mA (Li et al, 2021b), LA6mA, AL6mA (Zhang et al, 2021), GC6mA-Pred (Cai et al, 2022), Meta-i6mA (Hasan et al, 2021), and BERT6mA (Tsukiyama et al, 2022). Based on neural networks, Yu and Dai.…”

“…For example, Deep6mA presents an accuracy of more than 90% in predicting plants such as Arabidopsis (Li et al, 2021b). LA6mA and AL6mA capture location information from DNA sequences through a self-attention mechanism (Zhang et al, 2021). GC6mA-Pred mainly identifies 6 mA sites in the rice genome and outperforms several prediction models, including DNA6MA-MINT, on independent datasets (Cai et al, 2022).…”

BiLSTM-5mC: A Bidirectional Long Short-Term Memory-Based Approach for Predicting 5-Methylcytosine Sites in Genome-Wide DNA Promoters

“…DNA sequence analysis is similar to natural language processing, in which recurrent neural networks (RNNs) can be used to process the sequential data. As a popular and powerful RNN architecture, LSTM has been widely used to solve the problem of biological sequence analysis and has achieved excellent performance [43][44][45]. BiLSTM consists of two reversed unidirectional LSTM networks, which is a special type of RNN.…”