Recommendations for evaluation of computational methods

Abstract

The field of computational chemistry, particularly as applied to drug design, has become increasingly important in terms of the practical application of predictive modeling to pharmaceutical research and development. Tools for exploiting protein structures or sets of ligands known to bind particular targets can be used for bindingmode prediction, virtual screening, and prediction of activity. A serious weakness within the field is a lack of standards with respect to quantitative evaluation of methods, data set preparation, and data set sharing. Our goal should be to report new methods or comparative evaluations of methods in a manner that supports decision making for practical applications. Here we propose a modest beginning, with recommendations for requirements on statistical reporting, requirements for data sharing, and best practices for benchmark preparation and usage.

Retrospective Validation of a Structure-Based Virtual Screening Protocol to Identify Ligands for Estrogen Receptor Alpha and Its Application to Identify the Alpha-Mangostin Binding Pose

“…The ideal value of the AUC is 100% [26], which indicates that all known ligands are ranked higher than their decoys. In random sampling, the AUC value is 50% [32]. The EF 1% value represents the early enrichment of the protocols, while the AUC value represents the global enrichment [26,32].…”

Molecular Docking and Structure-Based Drug Design Strategies

“…Next, the selected structure undergoes several procedures to properly prepare it for the molecular docking studies. In short, the preparation routine consists of adding hydrogen atoms, eliminating water molecules (with the exception of those mediating important interactions), specifying the correct protonation and tautomerization states of the binding site residues, and calculating partial charges [172].…”

“…[175,176] PubChem [177,178] ChemSpider [179,180] ChEMBL [181,182] NuBBE DB [183,184] ChemBank [185,186] eMolecules [187] DrugBank [188,189] Binding DB [190,191] In a following step, the prepared database is docked into the target binding site. The conformational search algorithm explores the energy landscape of each molecule and high-scoring compounds are selected as potential ligands [169][170][171][172]. As virtual screening involves hundreds of thousands (or millions) of compounds, post-docking analysis is usually conducted to decide which compounds to prioritize.…”

Conformal prediction of biological activity of chemical compounds

“…It has to be noted at this stage that there does not seem to be an established consensus on what the best performance criteria are in the domain of Compound Activity Prediction (see for instance [12]), although Precision (fraction of actual Actives among compounds predicted as Active) and Recall (fraction of all the Active compounds that are among those predicted as Active) seem to be generally relevant. In addition, it is worth pointing out that these (and many other) criteria of performance should be considered as generalisations of classical performance criteria since they include dependence of the results on the required confidence level.…”