LMPhosSite: A Deep Learning-Based Approach for General Protein Phosphorylation Site Prediction Using Embeddings from the Local Window Sequence and Pretrained Protein Language Model

Abstract

Phosphorylation is one of the most important post-translational modifications and plays a pivotal role in various cellular processes. Although there exist several computational tools to predict phosphorylation sites, existing tools have not yet harnessed the knowledge distilled by pretrained protein language models. Herein, we present a novel deep learning-based approach called LMPhosSite for the general phosphorylation site prediction that integrates embeddings from the local window sequence and the contextualized embedding obtained using global (overall) protein sequence from a pretrained protein language model to improve the prediction performance. Thus, the LMPhosSite consists of two base-models: one for capturing effective local representation and the other for capturing global per-residue contextualized embedding from a pretrained protein language model. The output of these base-models is integrated using a score-level fusion approach. LMPhosSite achieves a precision, recall, Matthew's correlation coefficient, and F1-score of 38.78%, 67.12%, 0.390, and 49.15%, for the combined serine and threonine independent test data set and 34.90%, 62.03%, 0.298, and 44.67%, respectively, for the tyrosine independent test data set, which is better than the compared approaches. These results demonstrate that LMPhosSite is a robust computational tool for the prediction of the general phosphorylation sites in proteins.

A study on experimental bias in post-translational modification predictors

“…These predictors traditionally use one-hot encoded representations, with a combination of a CNN module and a capsule network module (MusiteDeep 18 ), three densely connected CNN blocks (DeepPhos 19 ), or a local context module and a global module built from convolutional and LSTM layers, and a squeeze-and-excitation network (DeepPSP 13 ). Finally, LMPhosSite 28 uses a CNN module on a trainable embedding, in combination with a fully-connected neural network built on the pLM representation for the single phosphosite position. For more information, see the Methods section, and their respective publications.…”

“…Compared to the LMPhosSite tool that also employs ProtT5-XL-U50 based representations but with a different model, our CNN model showed an improvement of 2.3% to 8.9% in terms of AUPRC, except again on multi-source-Y, where performance was roughly equal. 39 , data from multiple traditional sources (UniProtKB/SwissProt, dbPTM 40 , Phospho.ELM 41 , and PhosphoSitePlus 42 ) used as the benchmark in other phosphosite prediction publications 13,28 , and combined S/T phosphosites from individual experiments with different proteases 1 . Average curves over 10 runs (per dataset) are depicted.…”

“…These classical approaches require numerical feature vector representations engineered from the amino acids surrounding a candidate phosphosite, which we will refer to as the receptive field. Manually crafted input features for phosphorylation prediction models include physicochemical properties 14,16,21,26 , information theory features 15,17,28 , as well as additional information such as structural features (e.g., secondary structure) 10,16,24 , functional features 10,24 , and protein-protein interactions 17,25 .…”

“…These models learn low-level feature representations from the sequence input data during training, alleviating the need for preliminary manual feature engineering. Recently, LMPhosSite utilized protein language models (pLMs) to improve phosphosite prediction by adding single-position embeddings as input features 28 . PLMs are pretrained models that yield enriched, structure-aware sequence representations, instead of merely encoding the amino acid composition of a receptive field in a protein [29][30][31][32][33][34][35] .…”

HumanO-linked Glycosylation Site Prediction Using Pretrained Protein Language Model

“…An important process while developing a ML/DL model for the prediction of protein O-linked glycosylation sites is representing the primary amino acids with fixed size pertinent feature embeddings via a suitable encoding scheme 48,58 . We employed a pretrained PLM to generate per-residue contextualized embeddings for this investigation.…”