Improving language model of human genome for DNA–protein binding prediction based on task-specific pre-training

Luo, Hanyu; Shan, Wenyu; Chen, Cheng; Ding, Pingjian; Luo, Lingyun

doi:10.1007/s12539-022-00537-9

Cited by 5 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, deep learning methods have made significant advancements in motif mining. There are three commonly used types of deep learning methods: convolutional neural network (CNN) 12 , 13 , recurrent neural networks (RNN) 14 – 16 , and hybrid CNN-RNN 17 . These techniques have been enhanced to autonomously predict and recognize motifs, resulting in better predictions based on the provided data.…”

Section: Introductionmentioning

confidence: 99%

GMean—a semi-supervised GRU and K-mean model for predicting the TF binding site

Chuah,

He,

Huang

2024

Sci Rep

View full text Add to dashboard Cite

The transcription factor binding site is a deoxyribonucleic acid sequence that binds to transcription factors. Transcription factors are proteins that regulate the transcription gene. Abnormal turnover of transcription factors can lead to uncontrolled cell growth. Therefore, discovering the relationships between transcription factors and deoxyribonucleic acid sequences is an important component of bioinformatics research. Numerous deep learning and machine learning language models have been developed to accomplish these tasks. Our goal in this work is to propose a GMean model for predicting unlabelled deoxyribonucleic acid sequences. The GMean model is a hybrid model with a combination of gated recurrent unit and K-mean clustering. The GMean model is developed in three phases. The labelled and unlabelled data are processed based on k-mers and tokenization. The labelled data is used for training. The unlabelled data are used for testing and prediction. The experimental data consists of deoxyribonucleic acid experimental of GM12878, K562 and HepG2. The experimental results show that GMean is feasible and effective in predicting deoxyribonucleic acid sequences, as the highest accuracy is 91.85% in predicting K562 and HepG2. This is followed by the prediction of the sequence between GM12878 and K562 with an accuracy of 89.13%. The lowest accuracy is the prediction of the sequence between HepG2 and GM12828, which is 88.80%.

show abstract

Section: Introductionmentioning

confidence: 99%

GMean—a semi-supervised GRU and K-mean model for predicting the TF binding site

Chuah,

He,

Huang

2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…[8], [9], natural language processing [10], [11], bioinformatics [12], [13] and image analysis [14], [15]. Methods based on convolutional neural networks (CNN) [16] and recurrent neural networks (RNN) [17], [18] like gated recurrent unit (GRU), long short-term memory networks (LSTM) have been proposed to analyse and predict genome DNA. These techniques have been improved to generate autonomous prediction at learning process that spot specific trends and patterns to make better decisions based on the given data.…”

Section: Introductionmentioning

confidence: 99%

DeepBiG: A Hybrid Supervised CNN and Bidirectional GRU Model for Predicting the DNA Sequence

Chuah,

He,

Huang

2024

IJACSA

View full text Add to dashboard Cite

Understanding the deoxyribonucleic acid (DNA) sequence is a major component of bioinformatics research. The amount of biological data increases tremendously. Hence, there is a need for effective approaches to handle the critical problem in the general computational framework of DNA sequence prediction and classification. Numerous deep learning languages can be used to complete these tasks compared to manual techniques that have been followed for ages. The aim of this project is to employ effective approaches for pre-processing DNA sequences and using deep learning languages to train the sequences for making judgments, predictions, and classifications of DNA sequences into known categories. In this study, the pre-processing methods include k-mers and tokenization. We employ a novel hybrid deep learning algorithm that combines convolutional neural networks and is followed by bidirectional gated recurrent networks. This combination can capture dependencies within the genome sequence, even in large datasets with a lot of noise. The proposed model is compared with existing widely used models and classifiers. The results show that the proposed model achieves a good result with an accuracy of 82.90%. The dataset consists of 44,391 labeled DNA sequences obtained from the Encode project.

show abstract

“…[27, 6, 19] Numerous studies have explored various aspects not only from the perspective of computer science like medical text analysis [16, ? ], public health data mining [25] and GNN benchmarking [14] but also biological and medical like KVPLM [26], DNA-Pretrain [18], proteome analysis [15, 20, 11], and peptide property prediction [10] within the realm of bioinformatics. Nevertheless, the perspective of LLMs has predominantly focused on handling medical texts, making it challenging for generative artificial intelligence to find suitable entry points and engage effectively.…”

Section: Introductionmentioning

confidence: 99%

Bioinfo-Bench: A Simple Benchmark Framework for LLM Bioinformatics Skills Evaluation

Chen,

Deng

2023

Preprint

View full text Add to dashboard Cite

Large Language Models (LLMs) have garnered significant recognition in the life sciences for their capacity to comprehend and utilize knowledge. The contemporary expectation in diverse industries extends beyond employing LLMs merely as chatbots; instead, there is a growing emphasis on harnessing their potential as adept analysts proficient in dissecting intricate issues within these sectors. The realm of bioinformatics is no exception to this trend. In this paper, we introduce Bioinfo-Bench, a novel yet straightforward benchmark framework suite crafted to assess the academic knowledge and data mining capabilities of foundational models in bioinformatics. Bioinfo-Bench systematically gathered data from three distinct perspectives: knowledge acquisition, knowledge analysis, and knowledge application, facilitating a comprehensive examination of LLMs. Our evaluation encompassed prominent models ChatGPT, Llama, and Galactica. The findings revealed that these LLMs excel in knowledge acquisition, drawing heavily upon their training data for retention. However, their proficiency in addressing practical professional queries and conducting nuanced knowledge inference remains constrained. Given these insights, we are poised to delve deeper into this domain, engaging in further extensive research and discourse. It is pertinent to note that project Bioinfo-Bench is currently in progress, and all associated materials will be made publicly accessible.

show abstract

Improving language model of human genome for DNA–protein binding prediction based on task-specific pre-training

Cited by 5 publications

References 34 publications

GMean—a semi-supervised GRU and K-mean model for predicting the TF binding site

GMean—a semi-supervised GRU and K-mean model for predicting the TF binding site

DeepBiG: A Hybrid Supervised CNN and Bidirectional GRU Model for Predicting the DNA Sequence

Bioinfo-Bench: A Simple Benchmark Framework for LLM Bioinformatics Skills Evaluation

Contact Info

Product

Resources

About