The Community Structure–Activity Resource (CSAR) recently held its first blinded exercise based on data provided by Abbott, Vertex, and colleagues at the University of Michigan, Ann Arbor. A total of 20 research groups submitted results for the benchmark exercise where the goal was to compare different improvements for pose prediction, enrichment, and relative ranking of congeneric series of compounds. The exercise was built around blinded high-quality experimental data from four protein targets: LpxC, Urokinase, Chk1, and Erk2. Pose prediction proved to be the most straightforward task, and most methods were able to successfully reproduce binding poses when the crystal structure employed was co-crystallized with a ligand from the same chemical series. Multiple evaluation metrics were examined, and we found that RMSD and native contact metrics together provide a robust evaluation of the predicted poses. It was notable that most scoring functions underpredicted contacts between the hetero atoms (i.e., N, O, S, etc.) of the protein and ligand. Relative ranking was found to be the most difficult area for the methods, but many of the scoring functions were able to properly identify Urokinase actives from the inactives in the series. Lastly, we found that minimizing the protein and correcting histidine tautomeric states positively trended with low RMSD for pose prediction but minimizing the ligand negatively trended. Pregenerated ligand conformations performed better than those that were generated on the fly. Optimizing docking parameters and pretraining with the native ligand had a positive effect on the docking performance as did using restraints, substructure fitting, and shape fitting. Lastly, for both sampling and ranking scoring functions, the use of the empirical scoring function appeared to trend positively with the RMSD. Here, by combining the results of many methods, we hope to provide a statistically relevant evaluation and elucidate specific shortcomings of docking methodology for the community.

A major goal in drug design is the improvement of computational methods for docking and scoring. The Community Structure Activity Resource (CSAR) has collected several data sets from industry and added in-house data sets that may be used for this purpose (). CSAR has currently obtained data from Abbott, GlaxoSmithKline, and Vertex and is working on obtaining data from several others. Combined with our in-house projects, we are providing a data set consisting of 6 protein targets, 647 compounds with biological affinities, and 82 crystal structures. Multiple congeneric series are available for several targets with a few representative crystal structures of each of the series. These series generally contain a few inactive compounds, usually not available in the literature, to provide an upper bound to the affinity range. The affinity ranges are typically 3–4 orders of magnitude per series. For our in-house projects, we have had compounds synthesized for biological testing. Affinities were measured by Thermofluor, Octet RED, and isothermal titration calorimetry for the most soluble. This allows the direct comparison of the biological affinities for those compounds, providing a measure of the variance in the experimental affinity. It appears that there can be considerable variance in the absolute value of the affinity, making the prediction of the absolute value ill-defined. However, the relative rankings within the methods are much better, and this fits with the observation that predicting relative ranking is a more tractable problem computationally. For those in-house compounds, we also have measured the following physical properties: logD, logP, thermodynamic solubility, and pKa. This data set also provides a substantial decoy set for each target consisting of diverse conformations covering the entire active site for all of the 58 CSAR-quality crystal structures. The CSAR data sets (CSAR-NRC HiQ and the 2012 release) provide substantial, publically available, curated data sets for use in parametrizing and validating docking and scoring methods.

The 2014 CSAR Benchmark Exercise was the last community-wide exercise that was conducted by the group at the University of Michigan, Ann Arbor. For this event, GlaxoSmithKline (GSK) donated unpublished crystal structures and affinity data from in-house projects. Three targets were used: tRNA (m1G37) methyltransferase (TrmD), Spleen Tyrosine Kinase (SYK), and Factor Xa (FXa). A particularly strong feature of the GSK data is its large size, which lends greater statistical significance to comparisons between different methods. In Phase 1 of the CSAR 2014 Exercise, participants were given several protein-ligand complexes and asked to identify the one near-native pose from among 200 decoys provided by CSAR. Though decoys were requested by the community, we found that they complicated our analysis. We could not discern whether poor predictions were failures of the chosen method or an incompatibility between the participant’s method and the setup protocol we used. This problem is inherent to decoys and we strongly advise against their use. In Phase 2, participants had to dock and rank/score a set of small molecules given only the SMILES strings of the ligands and a protein structure with a different ligand bound. Overall, docking was a success for most participants, much better in Phase 2 than in Phase 1. However, scoring was a greater challenge. No particular approach to docking and scoring had an edge, and successful methods included empirical, knowledge-based, machine-learning, shape-fitting, and even those with solvation and entropy terms. Several groups were successful in ranking TrmD and/or SYK, but ranking FXa ligands was intractable for all participants. Methods that were able to dock well across all submitted systems include MDock1, Glide-XP2, PLANTS3, Wilma4, Gold5, SMINA6, Glide-XP2/PELE7, FlexX8, and MedusaDock9. In fact, the submission based on Glide-XP2/PELE7 cross-docked all ligands to many crystal structures, and it was particularly impressive to see success across an ensemble of protein structures for multiple targets. For scoring/ranking, submissions that showed statistically significant achievement include MDock1 using ITScore1,10 with a flexible-ligand term11, SMINA6 using Autodock-Vina12,13, FlexX8 using HYDE14, and Glide-XP2 using XP DockScore2 with and without ROCS15 shape similarity16. Of course, these results are for only three protein targets, and many more systems need to be investigated to truly identify which approaches are more successful than others. Furthermore, our exercise is not a competition.

The stimulator-of-interferon-genes (STING) protein is involved in innate immunity. It has recently been shown that modulation of STING can lead to an aggressive antitumor response. DMXAA is an antitumor agent that had shown great promise in murine models but failed in human clinical trials. The molecular target of DMXAA was subsequently shown to be murine STING (mSTING); however, human STING (hSTING) is insensitive to DMXAA. Molecular dynamics simulations were employed to investigate the differences between hSTING and mSTING that could influence DMXAA binding. An initial set of simulations was performed to investigate a single lid region mutation G230I in hSTING (corresponding residue in mSTING is an Ile), which rendered the protein sensitive to DMXAA. The simulations found that an Ile side chain was enough to form a steric barrier that prevents exit of DMXAA, whereas in WT hSTING, the Gly residue that lacks a side chain formed a porous lid region that allowed DMXAA to exit. A second set of molecular dynamics simulations compared the tendency of STING to be in an open-inactive conformation or a closed-active conformation. The results show that hSTING prefers to be in an open-inactive conformation even with cGAMP, the native ligand, bound. On the other hand, mSTING prefers a closed-active conformation even without a ligand bound. These results highlight the challenges in translating a mouse active STING compound into a human active compound, while also providing avenues to pursue for designing a small-molecule drug targeting human STING.

Community Structure-Activity Resource (CSAR) conducted a benchmark exercise to evaluate the current computational methods for protein design, ligand docking, and scoring/ranking. The exercise consisted of three phases. The first phase required the participants to identify and rank order which designed sequences were able to bind the small molecule digoxigenin. The second phase challenged the community to select a near-native pose of digoxigenin from a set of decoy poses for two of the designed proteins. The third phase investigated the ability of current methods to rank/score the binding affinity of 10 related steroids to one of the designed proteins (pKd = 4.1 to 6.7). We found that 11 of 13 groups were able to correctly select the sequence that bound digoxigenin, with most groups providing the correct three-dimensional structure for the backbone of the protein as well as all atoms of the active-site residues. Eleven of the 14 groups were able to select the appropriate pose from a set of plausible decoy poses. The ability to predict absolute binding affinities is still a difficult task, as 8 of 14 groups were able to correlate scores to affinity (Pearson-r > 0.7) of the designed protein for congeneric steroids and only 5 of 14 groups were able to correlate the ranks of the 10 related ligands (Spearman-ρ > 0.7).

At the onset of a drug discovery program, the goal is to identify novel compounds with appropriate chemical features that can be taken forward as lead series. Here, we describe three prospective case studies, Bruton Tyrosine Kinase (BTK), RAR-Related Orphan Receptor γ t (RORγt), and Human Leukocyte Antigen DR isotype (HLA-DR) to illustrate the positive impact of high throughput virtual screening (HTVS) on the successful identification of novel chemical series. Each case represents a project with a varying degree of difficulty due to the amount of structural and ligand information available internally or in the public domain to utilize in the virtual screens. We show that HTVS can be effectively employed to identify a diverse set of potent hits for each protein system even when the gold standard, high resolution structural data or ligand binding data for benchmarking, is not available.

Here, we report a high-throughput virtual screening (HTVS) study using phosphoinositide 3-kinase (both PI3Kγ and PI3Kδ). Our initial HTVS results of the Janssen corporate database identified small focused libraries with hit rates at 50% inhibition showing a 50-fold increase over those from a HTS (high-throughput screen). Further, applying constraints based on "chemically intuitive" hydrogen bonds and/or positional requirements resulted in a substantial improvement in the hit rates (versus no constraints) and reduced docking time. While we find that docking scoring functions are not capable of providing a reliable relative ranking of a set of compounds, a prioritization of groups of compounds (e.g., low, medium, and high) does emerge, which allows for the chemistry efforts to be quickly focused on the most viable candidates. Thus, this illustrates that it is not always necessary to have a high correlation between a computational score and the experimental data to impact the drug discovery process.

No abstract