Twenty years of bioinformatics research for protease-specific substrate and cleavage site prediction: a comprehensive revisit and benchmarking of existing methods

Abstract

The roles of proteolytic cleavage have been intensively investigated and discussed during the past two decades. This irreversible chemical process has been frequently reported to influence a number of crucial biological processes (BPs), such as cell cycle, protein regulation and inflammation. A number of advanced studies have been published aiming at deciphering the mechanisms of proteolytic cleavage. Given its significance and the large number of functionally enriched substrates targeted by specific proteases, many computational approaches have been established for accurate prediction of protease-specific substrates and their cleavage sites. Consequently, there is an urgent need to systematically assess the state-of-the-art computational approaches for protease-specific cleavage site prediction to further advance the existing methodologies and to improve the prediction performance. With this goal in mind, in this article, we carefully evaluated a total of 19 computational methods (including 8 scoring function-based methods and 11 machine learning-based methods) in terms of their underlying algorithm, calculated features, performance evaluation and software usability. Then, extensive independent tests were performed to assess the robustness and scalability of the reviewed methods using our carefully prepared independent test data sets with 3641 cleavage sites (specific to 10 proteases). The comparative experimental results demonstrate that PROSPERous is the most accurate generic method for predicting eight protease-specific cleavage sites, while GPS-CCD and LabCaS outperformed other predictors for calpain-specific cleavage sites. Based on our review, we then outlined some potential ways to improve the prediction performance and ease the computational burden by applying ensemble learning, deep learning, positive unlabeled learning and parallel and distributed computing techniques. We anticipate that our study will serve as a practical and useful guide for interested readers to further advance next-generation bioinformatics tools for protease-specific cleavage site prediction.

Software-aided workflow for predicting protease-specific cleavage sites using physicochemical properties of the natural and unnatural amino acids in peptide-based drug discovery: Peptide cleavage sites prediction workflow

“…To evaluate the performance of the selected classifier models for cleavage site prediction of multiple proteases, we carried out a 5-fold cross validation (5CV) test on each of selected proteases under investigation in this study. Moreover, the predictive performance on several 19 external datasets was explored. Firstly, we trained and compared the predictive performance of models with two learning methods LR and SVC using two sizes of local windows P1-P1' and P4-P4' For evaluation of predictive performance during 5CV test the following criteria were calculated: accuracy, ROC AUC, sensitivity and specificity.…”

“…These approaches use as an input data extracted from databases described above. Several reviews were published to compare developed models and their predictive performance [2,19,33]. These approaches can be classified in four main groups depending on the way how models were developed.…”

An Interpretable Prediction Model for Identifying N7-Methylguanosine Sites Based on XGBoost and SHAP

“…To evaluate the prediction performance of XG-m7G we used four metrics, that is, Sn, Sp, Acc, and MCC, which have previously been used to assess the performance of predictors in other studies. 33 , 34 We also used ROC curves, 35 , 36 , 37 , 38 which plot the true-positive rate against the false-positive rate, and AUC to further assess the model performance. Sn, Sp, Acc, and MCC are defined as follows: where represents the total number of m7G site-containing sequences, represents the total number of non-m7G sequences, represents the number of m7G site-containing sequences incorrectly predicted as non-m7G sequences, and represents the number of non-m7G sequences incorrectly predicted as m7G site-containing sequences.…”

Recent Progresses for Computationally Identifying N6-methyladenosine Sites in Saccharomyces cerevisiae

“…where + represents the total number of positive samples investigated, while − + is the number of positive samplesincorrectly predicted to be of negative one; − the total number of negative samples investigated, while + − the number of the negative samples incorrectly predicted to be of positive one. The set of intuitive metrics has been concurred and applauded by a series of recent publications (see, e.g., [14, 16, 57-59, 74, 82, 87-100] [83,[101][102][103][104][105][106][107][108][109][110][111][112][113][114]). It is instructive t point out, however, either the conventional metrics [84] taken from math books or the intuitive metrics of Eq.5 are valid only for single label systems (where each of the constituent samples belong to one, and only one, attribute or class); for the multi-label systems (where a sample may simultaneously belong to several different attributes or classes) whose existence has become more frequent in system biology [6,7,29,[115][116][117][118][119][120][121][122][123][124][125][126][127][128][129][130][131][132][133][134], system medicine [135,…”