Extending in Silico Protein Target Prediction Models to Include Functional Effects

Abstract

In silico protein target deconvolution is frequently used for mechanism-of-action investigations; however existing protocols usually do not predict compound functional effects, such as activation or inhibition, upon binding to their protein counterparts. This study is hence concerned with including functional effects in target prediction. To this end, we assimilated a bioactivity training set for 332 targets, comprising 817,239 active data points with unknown functional effect (binding data) and 20,761,260 inactive compounds, along with 226,045 activating and 1,032,439 inhibiting data points from functional screens. Chemical space analysis of the data first showed some separation between compound sets (binding and inhibiting compounds were more similar to each other than both binding and activating or activating and inhibiting compounds), providing a rationale for implementing functional prediction models. We employed three different architectures to predict functional response, ranging from simplistic random forest models (‘Arch1’) to cascaded models which use separate binding and functional effect classification steps (‘Arch2’ and ‘Arch3’), differing in the way training sets were generated. Fivefold stratified cross-validation outlined cascading predictions provides superior precision and recall based on an internal test set. We next prospectively validated the architectures using a temporal set of 153,467 of in-house data points (after a 4-month interim from initial data extraction). Results outlined Arch3 performed with the highest target class averaged precision and recall scores of 71% and 53%, which we attribute to the use of inactive background sets. Distance-based applicability domain (AD) analysis outlined that Arch3 provides superior extrapolation into novel areas of chemical space, and thus based on the results presented here, propose as the most suitable architecture for the functional effect prediction of small molecules. We finally conclude including functional effects could provide vital insight in future studies, to annotate cases of unanticipated functional changeover, as outlined by our CHRM1 case study.

Comparison of Scaling Methods to Obtain Calibrated Probabilities of Activity for Protein–Ligand Predictions

“…21 More specifically, published inhouse models were trained using a very different number of "binding" or "functionally active" compounds (pXC 50 better than 5.0 (10 μM) at a target) across the models, with a median of 752 training instances and a (∼6.5 times larger) standard deviation of 4954 training points per protein target model. 22 Benchmark results highlighted models trained with the fewest data points assigned significantly lower probability estimates (frequently below 0.5) because of the narrow coverage of chemical space. Training set size implicitly affects probability calibration performance between protein families to a disproportionate extent for this reason because enzymes and kinases dominate protein family distributions (∼30% and ∼34% of the training data set size 23 ).…”

“…For example, bioactivity data (compound–protein target associations) in chemogenomic repositories comprise very different data distributions across the experimental data available (i.e., highly different numbers of compound annotations, target family classes, and the degree of diversity in the chemical matter tested) . More specifically, published in-house models were trained using a very different number of “binding” or “functionally active” compounds (pXC 50 better than 5.0 (10 μM) at a target) across the models, with a median of 752 training instances and a (∼6.5 times larger) standard deviation of 4954 training points per protein target model . Benchmark results highlighted models trained with the fewest data points assigned significantly lower probability estimates (frequently below 0.5) because of the narrow coverage of chemical space.…”

A Comparison of Scaling Methods to Obtain Calibrated Probabilities of Activity for Ligand-Target Predictions

“…different target classes, number of compounds, with higher or lower diversity) 18 . For example, published models using in-house data comprised a very different number of active compounds across the proteins modelled, with a median of 752 and (~6.5 times larger) standard deviation of 4,954 compounds per protein target model 19 . Another study modelled 15 different protein families, with a range of 17 to 615 targets per family, and (comparatively large) standard deviation of 174 targets across families 20 .…”