Identification of Thermophilic Proteins Based on Sequence-Based Bidirectional Representations from Transformer-Embedding Features

Pei, Hongdi; Li, Jiayu; Ma, Shuhan; Jiang, Jici; Li, Mingxin; Zou, Quan; Lv, Zhibin

doi:10.3390/app13052858

Cited by 12 publications

(9 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To compare TemStaPro with modern deep learning-based methods, it was tested on the iThermo ( Ahmed et al 2022 ) test dataset. The DeepTP ( Zhao et al 2023 ) and BertThermo ( Pei et al 2023 ) performance results were taken from the BertThermo publication. Among BertThermo, DeepTP, and iThermo, BertThermo achieved the best evaluation classification scores.…”

Section: Resultsmentioning

confidence: 99%

“…It was only a matter of time before pLM embeddings were also applied for the identification of thermostable proteins. To the best of our knowledge, BertThermo ( Pei et al 2023 ) was the first such method that was published. However, BertThermo was trained using only 2803 sequences and was based on a binary classifier for only a single temperature threshold.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

TemStaPro: protein thermostability prediction using sequence representations from protein language models

Pudžiuvelytė,

Olechnovič,

Godliauskaite

et al. 2024

Bioinformatics

View full text Add to dashboard Cite

Motivation Reliable prediction of protein thermostability from its sequence is valuable for both academic and industrial research. This prediction problem can be tackled using machine learning and by taking advantage of the recent blossoming of deep learning methods for sequence analysis. These methods can facilitate training on more data and, possibly, enable development of more versatile thermostability predictors for multiple ranges of temperatures. Results We applied the principle of transfer learning to predict protein thermostability using embeddings generated by protein language models (pLMs) from an input protein sequence. We used large pLMs that were pre-trained on hundreds of millions of known sequences. The embeddings from such models allowed us to efficiently train and validate a high-performing prediction method using over one million sequences that we collected from organisms with annotated growth temperatures. Our method, TemStaPro (Temperatures of Stability for Proteins), was used to predict thermostability of CRISPR-Cas Class II effector proteins (C2EPs). Predictions indicated sharp differences among groups of C2EPs in terms of thermostability and were largely in tune with previously published and our newly obtained experimental data. Availability and Implementation TemStaPro software and the related data are freely available from https://github.com/ievapudz/TemStaPro and https://doi.org/10.5281/zenodo.7743637.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

TemStaPro: protein thermostability prediction using sequence representations from protein language models

Pudžiuvelytė,

Olechnovič,

Godliauskaite

et al. 2024

Bioinformatics

View full text Add to dashboard Cite

show abstract

“…It integrates various deep learning methods into a self-contained protein T m predictor. In contrast to other predictors like SCooP, DeepTP, and BertThermo essentially classifying thermostable and non-thermostable proteins, Prostab2 and DeepSTABp both employ a regression model to predict continuous T m values [ 13 , 15 , 16 , 17 , 18 ]. Through proper sampling techniques, we trained the model for extensive prediction tasks and achieved superior results compared to the current state-of-the-art method, ProTstab2, by reducing the mean average prediction error by around 35 percent ( Table 1 , MAE).…”

Section: Discussionmentioning

confidence: 99%

“…Further, available algorithms for thermal stability prediction based on cell-wide analysis of protein stability TPP experiment differ regarding the definition of the learning problem. DeepTP [ 17 ] and BertThermo [ 18 ] approaches construct a classification problem to distinguish between thermostable and thermolabile proteins, but do not intend to predict T m values. Although classification-based predictors have demonstrated outstanding performance, they simplify the prediction task by reducing the number of output classes to a discrete set, whereas regression-based analysis captures the continuous nature of T m values.…”

Section: Introductionmentioning

confidence: 99%

DeepSTABp: A Deep Learning Approach for the Prediction of Thermal Protein Stability

Jung

Frey

Zimmer

et al. 2023

IJMS

View full text Add to dashboard Cite

Proteins are essential macromolecules that carry out a plethora of biological functions. The thermal stability of proteins is an important property that affects their function and determines their suitability for various applications. However, current experimental approaches, primarily thermal proteome profiling, are expensive, labor-intensive, and have limited proteome and species coverage. To close the gap between available experimental data and sequence information, a novel protein thermal stability predictor called DeepSTABp has been developed. DeepSTABp uses a transformer-based protein language model for sequence embedding and state-of-the-art feature extraction in combination with other deep learning techniques for end-to-end protein melting temperature prediction. DeepSTABp can predict the thermal stability of a wide range of proteins, making it a powerful and efficient tool for large-scale prediction. The model captures the structural and biological properties that impact protein stability, and it allows for the identification of the structural features that contribute to protein stability. DeepSTABp is available to the public via a user-friendly web interface, making it accessible to researchers in various fields.

show abstract

“…ProtTrans 35 , a family of models including protBERT, leverages transformers to extract protein characteristics from sequence data. BertThermo 36 uses the protBERT embeddings with classical machine learning models for thermophilicity classification, whereas DeepSTABp incorporates ProtTrans-XL embeddings and growth temperature to predict protein melting temperature 37 . Similarly, TemStaPro 38 is an ensemble of models incorporating ProtT5-XL 35 embeddings to feed-forward densely connected neural network models, and ProLaTherm 39 integrates the encoder part of a T5-3B 40 model with ProtT5-XL 35 as the feature extractor.…”

Section: Introductionmentioning

confidence: 99%

TemBERTure: Advancing protein thermostability prediction with Deep Learning and attention mechanisms

Rodella,

Lazaridi,

Lemmin

2024

Preprint

View full text Add to dashboard Cite

Understanding protein thermostability is essential for various biotechnological and biological applications. However, traditional experimental methods for assessing this property are time-consuming, expensive, and error prone. Recently, the application of Deep Learning techniques from Natural Language Processing (NLP) was extended to the field of biology, with an emphasis on protein modeling. From a linguistic perspective, the primary sequence of proteins can be viewed as a string of amino acids that follow a physicochemical grammar. This study explores the potential of Deep Learning models trained on protein sequences to predict protein thermostability which provide improvements with respect to current approaches. We implemented TemBERTure, a Deep Learning framework to classify the thermal class (non thermophilic or thermophilic) and predict and melting temperature of a protein, based on its primary sequence. Our findings highlight the critical role that data diversity plays on training robust models. Models trained on datasets with a wider range of sequences from various organisms exhibited superior performance compared to those with limited diversity. This emphasizes the need for a comprehensive data curation strategy that ensures a balanced representation of diverse species in the training data, to avoid the risk that the model focuses on recognizing the evolutionary lineage of the sequence rather than the intrinsic thermostability features. In order to gain more nuanced insights into protein thermostability, we propose leveraging attention scores within Deep Learning models to gain more nuanced insights into protein thermostability. We show that analyzing these scores alongside the 3D protein structure could offer a better understanding of the complex interplay between amino acid properties, their positioning, and the surrounding microenvironment, all crucial factors influencing protein thermostability. This work sheds light on the limitations of current protein thermostability prediction methods and introduces new avenues for exploration. By emphasizing data diversity and utilizing refined attention scores, future research can pave the way for more accurate and informative methods for predicting protein thermostability.

show abstract

Identification of Thermophilic Proteins Based on Sequence-Based Bidirectional Representations from Transformer-Embedding Features

Cited by 12 publications

References 60 publications

TemStaPro: protein thermostability prediction using sequence representations from protein language models

TemStaPro: protein thermostability prediction using sequence representations from protein language models

DeepSTABp: A Deep Learning Approach for the Prediction of Thermal Protein Stability

TemBERTure: Advancing protein thermostability prediction with Deep Learning and attention mechanisms

Contact Info

Product

Resources

About