iEnhancer-ELM: improve enhancer identification by extracting position-related multiscale contextual information based on enhancer language models

Abstract

Motivation Enhancers are important cis-regulatory elements that regulate a wide range of biological functions and enhance the transcription of target genes. Although many feature extraction methods have been proposed to improve the performance of enhancer identification, they can’t learn position-related multiscale contextual information from raw DNA sequences. Results In this paper, we propose a novel enhancer identification method (iEnhancer-ELM) based on BERT-like enhancer language models. iEnhancer-ELM tokenizes DNA sequences with multi-scale k-mers and extracts contextual information of different scale k-mers related with their positions via an multi-head attention mechanism. We first evaluate the performance of different scale k-mers, then ensemble them to improve the performance of enhancer identification. The experimental results on two popular benchmark datasets show that our model outperforms state-of-the-art methods. We further illustrate the interpretability of iEnhancer-ELM. For a case study, we discover 30 enhancer motifs via a 3-mer-based model, where 12 of motifs are verified by STREME and JASPAR, demonstrating our model has a potential ability to unveil the biological mechanism of enhancer. Availability The models and associated code are available at https://github.com/chen-bioinfo/iEnhancer-ELM Supplementary information Supplementary data are available at Bioinformatics Advances online.

High-Activity Enhancer Generation based on Feedback GAN with Domain Constraint and Curriculum Learning

“…To find motifs enriched in the high-activity enhancers, all generated enhancers are divided into three groups according to their activity, high-activity group with activity larger than 3.0, low-activity group with activity less than 0.0 and midactivity group with activity between them. STREME [22,23] was employed to find relatively enriched motifs in each group.…”

Predmoter—cross-species prediction of plant promoter and enhancer regions

“…The networks in these studies performed a fundamentally different predictive task than actual promoter sequence prediction. Meanwhile, recent enhancer predicting networks, like PREPRINT ( Osmala and Lähdesmäki 2020 ), the cross-species predicting CrepHAN ( Hong et al 2021 ) or iEnhancer-ELM ( Li et al 2023 ), were trained on experimentally verified enhancers. All these studies utilize human and/or other mammalian enhancers.…”

A deep learning model for DNA enhancer prediction based on nucleotide position aware feature encoding

“…leveraged DNA structural features, combining natural language processing, convolutional neural networks, and long short-term memory to accurately predict enhancers in genomic data, a model referred to as PEDH. 34 Furthermore, enhancer recognition methods, such as iEnhancer-ELM based on a BERT-like enhancer language model (DNABERT), 35 and iEnhancer-BERT, 36 a transfer learning approach based on pre-trained DNA language models, have also been introduced. In summary, there is a growing body of research exploring the application of machine learning methods in DNA enhancer prediction, demonstrating promising performance and significant progress.…”