Toward more realistic drug-target interaction predictions

Abstract

A number of supervised machine learning models have recently been introduced for the prediction of drug–target interactions based on chemical structure and genomic sequence information. Although these models could offer improved means for many network pharmacology applications, such as repositioning of drugs for new therapeutic uses, the prediction models are often being constructed and evaluated under overly simplified settings that do not reflect the real-life problem in practical applications. Using quantitative drug–target bioactivity assays for kinase inhibitors, as well as a popular benchmarking data set of binary drug–target interactions for enzyme, ion channel, nuclear receptor and G protein-coupled receptor targets, we illustrate here the effects of four factors that may lead to dramatic differences in the prediction results: (i) problem formulation (standard binary classification or more realistic regression formulation), (ii) evaluation data set (drug and target families in the application use case), (iii) evaluation procedure (simple or nested cross-validation) and (iv) experimental setting (whether training and test sets share common drugs and targets, only drugs or targets or neither). Each of these factors should be taken into consideration to avoid reporting overoptimistic drug–target interaction prediction results. We also suggest guidelines on how to make the supervised drug–target interaction prediction studies more realistic in terms of such model formulations and evaluation setups that better address the inherent complexity of the prediction task in the practical applications, as well as novel benchmarking data sets that capture the continuous nature of the drug–target interactions for kinase inhibitors.

Drug-target interaction prediction: A Bayesian ranking approach

“…In contrast, Kinase contains continuous values of binding affinity for drugtarget pairs. In order to produce a binary interaction matrix, we used the same cutoff threshold as Pahikkala et al [37]. Drug-to-drug similarities were computed based on the chemical structure of the compound via the SIMCOMP algorithm (GPCR, IC, NR and E datasets) or via the 2D Tanimoto coefficients (Kinase dataset).…”

Revisiting Cross-Validation of Drug Similarity Based Classifiers Using Paired Data

“…In light of previous works highlighting the perils of cross-validation using paired data [6,7], we recently investigated the effect of using drug-wise disjoint cross-validation in predicting drug-disease pairs, where none of the drugs in the training set appeared in the test set [8]. We showed that the prediction accuracy of the classifier drops dramatically under such cross-validation setting, suggesting that the existing approaches are prone to over-fitting due to the inherent relationships in the data.…”

“…The proposed models are typically benchmarked using cross-validation, in which the known drug-disease or drug-drug associations are split into training and test sets. Though these methods report areas under receiver operating characteristic (ROC) curves around 90% under cross-validation, their applicability in translational medicine and, thus, ability to reduce drug development costs has been controversial [2,4,5].In light of previous works highlighting the perils of cross-validation using paired data [6,7], we recently investigated the effect of using drug-wise disjoint cross-validation in predicting drug-disease pairs, where none of the drugs in the training set appeared in the test set [8]. We showed that the prediction accuracy of the classifier drops dramatically under such cross-validation setting, suggesting that the existing approaches are prone to over-fitting due to the inherent relationships in the data.…”

Large-Scale Prediction of Drug-Target Interaction: a Data-Centric Review

“…A potential DTI can be determined by the predicted class label. There are other possible solutions, and some recent reviews focused on the technical aspect of mathematical or statistical methods (12)(13)(14)(15)(16)(17). Regardless, prior knowledge of drugs, targets, and their interactions is required for all in silico method development.…”