ProkBERT Family: Genomic Language Models for Microbiome Applications

Abstract

Machine learning offers transformative capabilities in microbiology and microbiome analysis, deciphering intricate microbial interactions, predicting functionalities, and unveiling novel patterns in vast datasets. This enriches our comprehension of microbial ecosystems and their influence on health and disease. However, the integration of machine learning in these fields contends with issues like the scarcity of labeled datasets, the immense volume and complexity of microbial data, and the subtle interactions within microbial communities. Addressing these challenges, we introduce the ProkBERT model family. Built on transfer learning and self-supervised methodologies, ProkBERT models capitalize on the abundant available data, demonstrating adaptability across diverse scenarios. The models’ learned representations align with established biological understanding, shedding light on phylogenetic relationships. With the novel Local Context-Aware (LCA) tokenization, the ProkBERT family overcomes the context size limitations of traditional transformer models without sacrificing performance or the information rich local context. In bioinformatics tasks like promoter prediction and phage identification, ProkBERT models excel. For promoter predictions, the best performing model achieved an MCC of 0.74 forE. coliand 0.62 in mixed-species contexts. In phage identification, they all consistently outperformed tools like VirSorter2 and DeepVirFinder, registering an MCC of 0.85. Compact yet powerful, the ProkBERT models are efficient, generalizable, and swift. They cater to both supervised and unsupervised tasks, providing an accessible tool for the community. The models are available on GitHub and HuggingFace.

Scorpio : Enhancing Embeddings to Improve Downstream Analysis of DNA sequences

“…We initiated the evaluation by collecting promoter and non-promoter sequences from the ProkBERT dataset 31 . Prokaryotic promoter sequences typically span 81 base pairs.…”

“…In this study, we utilized the promoter dataset provided by Ligeti et al 31 for training and testing our promoter prediction models. The promoter dataset by Ligeti et al consists of experimentally validated promoter sequences primarily drawn from the Prokaryotic Promoter Database (PPD), which includes sequences from 75 different organisms.…”

BioInformatics Agent (BIA): Unleashing the Power of Large Language Models to Reshape Bioinformatics Workflow

“…This trend presents both significant opportunities and new challenges for researchers in the field of biological sciences. For instance, a specific workflow for the processing whole-genome sequencing (WGS) data, involves utilization of dozens of software tools 3 and necessitating researchers to possess fundamental skills in software installation, parameter invocation, and troubleshooting capabilities. Furthermore, the in-depth analyses demand proficiency in an expanded toolkit of coding and visualization.…”

“…Bioinformatics is an interdisciplinary discipline, by leveraging cutting-edge computational methodologies and algorithmic strategies, it plays an irreplaceable role in many fields such as biology [1], medical science [2], microbiology [3], etc. Bioinformatics fosters a holistic understanding of biological processes by facilitating the integration and interpretation of multi-level data, from genomics to transcriptomics, proteomics, metabolomics, and beyond [4].…”