Benchmarking of protein descriptor sets in proteochemometric modeling (part 2): modeling performance of 13 amino acid descriptor sets

Abstract

BackgroundWhile a large body of work exists on comparing and benchmarking descriptors of molecular structures, a similar comparison of protein descriptor sets is lacking. Hence, in the current work a total of 13 amino acid descriptor sets have been benchmarked with respect to their ability of establishing bioactivity models. The descriptor sets included in the study are Z-scales (3 variants), VHSE, T-scales, ST-scales, MS-WHIM, FASGAI, BLOSUM, a novel protein descriptor set (termed ProtFP (4 variants)), and in addition we created and benchmarked three pairs of descriptor combinations. Prediction performance was evaluated in seven structure-activity benchmarks which comprise Angiotensin Converting Enzyme (ACE) dipeptidic inhibitor data, and three proteochemometric data sets, namely (1) GPCR ligands modeled against a GPCR panel, (2) enzyme inhibitors (NNRTIs) with associated bioactivities against a set of HIV enzyme mutants, and (3) enzyme inhibitors (PIs) with associated bioactivities on a large set of HIV enzyme mutants.ResultsThe amino acid descriptor sets compared here show similar performance (<0.1 log units RMSE difference and <0.1 difference in MCC), while errors for individual proteins were in some cases found to be larger than those resulting from descriptor set differences ( > 0.3 log units RMSE difference and >0.7 difference in MCC). Combining different descriptor sets generally leads to better modeling performance than utilizing individual sets. The best performers were Z-scales (3) combined with ProtFP (Feature), or Z-Scales (3) combined with an average Z-Scale value for each target, while ProtFP (PCA8), ST-Scales, and ProtFP (Feature) rank last.ConclusionsWhile amino acid descriptor sets capture different aspects of amino acids their ability to be used for bioactivity modeling is still – on average – surprisingly similar. Still, combining sets describing complementary information consistently leads to small but consistent improvement in modeling performance (average MCC 0.01 better, average RMSE 0.01 log units lower). Finally, performance differences exist between the targets compared thereby underlining that choosing an appropriate descriptor set is of fundamental for bioactivity modeling, both from the ligand- as well as the protein side.

Prediction of PARP Inhibition with Proteochemometric Modelling and Conformal Prediction

“…We found that all descriptors, with the exception of SOCN, performed at the same level of statistical significance (ANOVA P >0.05; Tukey’s Honest Significance Difference (HSD), α =0.05, n =15) (Table 1 and Figure 2A). These results are in agreement with previous studies, where model performance did not consistently vary across amino acid descriptor sets, and where prediction errors for individual targets were higher than the error differences obtained with models trained on different combinations of amino acid descriptors 26,30,32. Therefore, we conclude that the predictive signal provided by all protein descriptors except SOCN for the modelling of PARP inhibition is comparable.…”

“…To determine which protein descriptors provide the highest predictive signal, we benchmarked 8 binding site amino acid (Table 1A)29,30 and 11 full protein sequence descriptors31 (Table 1B). We trained 15 models for each combination of compound and protein descriptors, each time using different resamples to define the training and test sets.…”

Probing the origin of dihydrofolate reductase inhibition via proteochemometric modeling

“…PCM is a branch of chemometrics which uses mathematical and statistical approaches to model the interactions between a series of ligands and a set of receptors. One major strength of PCM is that it does not require structural information of proteins to provide specific information about their functions . Since its first introduction by Lapinsh et al in 2001, the approach has been successfully applied to investigate different protein families such as cytochrome P450, kinases, melanocortin receptors, G protein‐coupled receptors, HIV proteases, aromatases, carbonic anhydrases, and phosphodiesterases .…”

Proteochemometric Modeling of the Interaction Space of Carbonic Anhydrase and its Inhibitors: An Assessment of Structure‐based and Sequence‐based Descriptors

“…Unlike the traditional QSAR, in proteochemometric modeling (PCM) approach descriptors of proteins and cross‐terms made from descriptors of ligands and proteins are correlated with the activity data for protein–ligand interactions . A recent study has revealed that combinations of descriptors from different aspect may help increase the performance of proteochemometric modeling .…”