Deciphering microbial gene function using natural language processing

Miller, Danielle; Stern, Adi; Burstein, David

doi:10.1038/s41467-022-33397-4

Cited by 22 publications

(24 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fourth, adding non-protein modalities (e.g. noncoding regulatory elements 13 ) as input to gLM may also greatly improve gLM's representation of biological sequence data, and can learn protein function and regulation conditioned upon other modalities 47 . One of the most powerful aspects of the transformer-based language models is their potential for transfer learning and fine-tuning.…”

Section: Discussionmentioning

confidence: 99%

“…Thus, there exists an inherent evolutionary linkage between genes, their genomic context, and gene function [13][14][15] , which can be explored by characterizing patterns that emerge from large metagenomic datasets. Recent efforts to model genomic information have shown predictive power of genomic context in gene function 16 and metabolic trait evolution 17 in bacterial and archaeal genomes. However, these methods represent genes as categorical entities, despite these genes existing in continuous space where multidimensional properties such as phylogeny, structure, and function are abstracted in their sequences.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Genomic language model predicts protein co-regulation and function

Hwang

Cornman²,

Овчинников

et al. 2023

Preprint

View full text Add to dashboard Cite

Deciphering the relationship between a gene and its genomic context is fundamental to understanding and engineering biological systems. Machine learning has shown promise in learning latent relationships underlying the sequence-structure-function paradigm from massive protein sequence datasets. However, to date, limited attempts have been made in extending this continuum to include higher order genomic context information. Evolutionary processes dictate the specificity of genomic contexts in which a gene is found across phylogenetic distances, and these emergent genomic patterns can be leveraged to uncover functional relationships between gene products. Here, we trained a genomic language model (gLM) on millions of metagenomic scaffolds to learn the latent functional and regulatory relationships between genes. gLM learns contextualized protein embeddings that capture the genomic context as well as the protein sequence itself, and appears to encode biologically meaningful and functionally relevant information (e.g. phylogeny, enzymatic function). Our analysis of the attention patterns demonstrates that gLM is learning co-regulated functional modules (i.e. operons). Our findings illustrate that gLM's unsupervised deep learning of the metagenomic corpus is an effective approach to encode functional semantics and regulatory syntax of genes in their genomic contexts, providing a promising avenue for uncovering complex relationships between genes in a genomic region.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Genomic language model predicts protein co-regulation and function

Hwang

Cornman²,

Овчинников

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…AI methods implemented range from deep learning 10 to natural language processing (NPL) tools that extract rules from the language of protein sequences in the form of vectorized embeddings (protein language models 11,12 ). In addition, some methods have been created to predict the function of specific proteins, such as enzymes 13 or prokaryotic viral proteins 14 , while others have been successfully used to predict non-GO term-based functions in a wider context 15,16 .…”

Section: Introductionmentioning

confidence: 99%

Illuminating the functional landscape of the dark proteome across the Animal Tree of Life through natural language processing models

Martínez-Redondo,

Barrios-Núñez,

Vázquez-Valls

et al. 2024

Preprint

View full text Add to dashboard Cite

Understanding how coding genes and their functions evolve over time is a key aspect of evolutionary biology. Protein coding genes poorly understood or characterized at the functional level may be related to important evolutionary innovations, potentially leading to incomplete or inaccurate models of evolutionary change, and limiting the ability to identify conserved or lineage-specific features. Homology-based methodologies often fail to transfer functional annotations in a large fraction of the coding gene repertoire in non-model organisms. This is particularly relevant in animals, where a high number of their coding genes yield no functional annotation. Here, we leverage machine learning and natural language processing models to investigate functional annotation in the ‘dark proteome’ (defined as the unknown functional landscape’) of ca. 1,000 gene repertoires of virtually all animal phyla, totaling ca. 23.2 million coding genes. Gene ontology annotations were transferred to virtually all genes, with the model ProtT5 outperforming both homology-based and other machine learning-based models. We then explored the ‘dark proteome’ of all animal phyla revealing an enrichment in functions related to immune response, viral infection, response to stimuli, development, or signaling, among others. Furthermore, we provide an open-access pipeline - FANTASIA - to implement and benchmark these methodologies in any dataset. Our results uncover the putative functions of poorly understood protein-coding genes across the Animal Tree of Life and contribute to a more comprehensive understanding of the molecular basis of animal evolution.

show abstract

“…Inspired by the parallel and efficient processing of information in the biological brain, artificial neural networks (ANNs) have received a lot of attention and research − and have already achieved tremendous results in fields such as autonomous vehicles, biomedicine, natural language processing, and intelligent terminals . Most ANNs encode information as real-valued vectors for computation rather than as electrical spikes like the human brain, which leads to energy inefficiencies in ANNs.…”

Section: Introductionmentioning

confidence: 99%

Efficient Spiking Neural Networks with Biologically Similar Lithium-Ion Memristor Neurons

Ke,

Pan,

Jin

et al. 2024

ACS Appl. Mater. Interfaces

View full text Add to dashboard Cite

Benefiting from the brain-inspired event-driven feature and asynchronous sparse coding approach, spiking neural networks (SNNs) are becoming a potentially energy-efficient replacement for conventional artificial neural networks. However, neuromorphic devices used to construct SNNs persistently result in considerable energy consumption owing to the absence of sufficient biological parallels. Drawing inspiration from the transport nature of Na + and K + in synapses, here, a Li-based memristor (Li x AlO y ) was proposed to emulate the biological synapse, leveraging the similarity of Li as a homologous main group element to Na and K. The Li-based memristor exhibits ∼8 ns ultrafast operating speed, 1.91 and 0.72 linearity conductance modulation, and reproducible switching behavior, enabled by lithium vacancies forming a conductive filament mechanism. Moreover, these memristors are capable of simulating fundamental behaviors of a biological synapse, including long-term potentiation and long-term depression behaviors. Most importantly, a thresholdtunable leaky integrate-and-fire (TT-LIF) neuron is built using Li x AlO y memristors, successfully integrating synaptic signals from both temporal and spatial levels and achieving an optimal threshold of SNNs. A computationally efficient TT-LIF-based SNN algorithm is also implemented for image recognition schemes, featuring a high recognition rate of 90.1% and an ultralow firing rate of 0.335%, which is 4 times lower than those of other memristor-based SNNs. Our studies reveal the ion dynamics mechanism of the Li x AlO y memristor and confirm its potential in rapid switching and the construction of SNNs.

show abstract

Deciphering microbial gene function using natural language processing

Cited by 22 publications

References 67 publications

Genomic language model predicts protein co-regulation and function

Genomic language model predicts protein co-regulation and function

Illuminating the functional landscape of the dark proteome across the Animal Tree of Life through natural language processing models

Efficient Spiking Neural Networks with Biologically Similar Lithium-Ion Memristor Neurons

Contact Info

Product

Resources

About