CSAR Benchmark Exercise 2011–2012: Evaluation of Results from Docking and Relative Ranking of Blinded Congeneric Series

Abstract

The Community Structure–Activity Resource (CSAR) recently held its first blinded exercise based on data provided by Abbott, Vertex, and colleagues at the University of Michigan, Ann Arbor. A total of 20 research groups submitted results for the benchmark exercise where the goal was to compare different improvements for pose prediction, enrichment, and relative ranking of congeneric series of compounds. The exercise was built around blinded high-quality experimental data from four protein targets: LpxC, Urokinase, Chk1, and Erk2. Pose prediction proved to be the most straightforward task, and most methods were able to successfully reproduce binding poses when the crystal structure employed was co-crystallized with a ligand from the same chemical series. Multiple evaluation metrics were examined, and we found that RMSD and native contact metrics together provide a robust evaluation of the predicted poses. It was notable that most scoring functions underpredicted contacts between the hetero atoms (i.e., N, O, S, etc.) of the protein and ligand. Relative ranking was found to be the most difficult area for the methods, but many of the scoring functions were able to properly identify Urokinase actives from the inactives in the series. Lastly, we found that minimizing the protein and correcting histidine tautomeric states positively trended with low RMSD for pose prediction but minimizing the ligand negatively trended. Pregenerated ligand conformations performed better than those that were generated on the fly. Optimizing docking parameters and pretraining with the native ligand had a positive effect on the docking performance as did using restraints, substructure fitting, and shape fitting. Lastly, for both sampling and ranking scoring functions, the use of the empirical scoring function appeared to trend positively with the RMSD. Here, by combining the results of many methods, we hope to provide a statistically relevant evaluation and elucidate specific shortcomings of docking methodology for the community.

TACI deficiency enhances antibody avidity and clearance of an intestinal pathogen

“…Bioluminescence imaging (BLI) was run at 25°C with Octet RED (ForteBio) (46)(47)(48). K D was deduced from association and dissociation rates of NP-specific Ig (in dilutions of 0.3-10 nM) and measured with super streptavidin sensors (ForteBio) loaded with biotinylated NP 4 -BSA (Biosearch Technologies).…”

Statistical analysis of EGFR structures’ performance in virtual screening

“…Pose prediction and enrichment are two popular indicators of docking performance. Usually binding poses are able to be reproduced precisely,[16] so here we focus on enrichment of known inhibitors that is not very successful in benchmark exercises. The discerning ability of a single EGFR structure is measured by the enrichment factor (EF) at the top 1%.…”

Pharmacophore modeling: advances, limitations, and current utility in drug discovery

“…Although this ranking can still be better than random, typically only a few compounds are selected from those scoring best, and many of them often turn out to be inactive. 73,74 Several options are available for combining dockingbased virtual screening with pharmacophore-based virtual screening:…”