CavBench: A benchmark for protein cavity detection methods

Abstract

Extensive research has been applied to discover new techniques and methods to model protein-ligand interactions. In particular, considerable efforts focused on identifying candidate binding sites, which quite often are active sites that correspond to protein pockets or cavities. Thus, these cavities play an important role in molecular docking. However, there is no established benchmark to assess the accuracy of new cavity detection methods. In practice, each new technique is evaluated using a small set of proteins with known binding sites as ground-truth. However, studies supported by large datasets of known cavities and/or binding sites and statistical classification (i.e., false positives, false negatives, true positives, and true negatives) would yield much stronger and reliable assessments. To this end, we propose CavBench, a generic and extensible benchmark to compare different cavity detection methods relative to diverse ground truth datasets (e.g., PDBsum) using statistical classification methods.

A Comprehensive Mapping of the Druggable Cavities within the SARS-CoV-2 Therapeutically Relevant Proteins by Combining Pocket and Docking Searches as Implemented in Pockets 2.0

“…In detail, FPocket alone was able to recognize one half of the correct pockets (20 out of 40, precision = 0.50), a performance which is in line with that reported by the above mentioned benchmarking study for FPocket when used without repetitions [10]. The performances of docking scores alone are slightly worse than those reached by FPocket (18 out of 40, precision = 0.45) even though it should be noted that docking simulations are not blindly performed on all the protein surfaces but they are focused on the pockets previously detected by FPocket and thus the docking performances unavoidably benefit (at some extent) of the encouraging results offered by FPocket.…”

“…Moreover, the increasing number of experimentally resolved protein structures allows a markedly more accurate validation of these methods [9]. By using a purposely collected cavity database, a recent benchmarking analysis compared some well-known approaches for mapping the protein cavities and revealed that FPocket and GaussianFinder are those offering the best performances, with FPocket being able to identify the highest number of correct cavities [10].…”

Predicting binding sites from unbound versus bound protein structures

“…This implies that, contrary to historical belief, apo structures can perform as well as, or better than, holo structures. Previous studies have been mixed on the issue of apo versus holo structures 5,6,22 , and the behavior described here is attributed specifically to our dataset. Our complete data for this manuscript is provided in the Supplementary Information covering all calculated P, R, F, and MCC value for all individual PDB structures (Table S1 in the Supplementary Information) and giving median MCC and median F for the whole protein families (Table S2).…”

“…Only Fpocket (p = 0.03) has a statistically significant (p < 0.05) correlation between predictive power and structure type (holo vs. apo), again suggesting that holo structures perform slightly better with this method, but the trend is weak. CavBench's assessment showed equal performance of Apo and Holo protein structures in non-redundant binding site detection for Fpocket and for Ghecom 22 .…”

“…The recently published CavBench 22 benchmarking tool is highly related to the work herein. Its CavDataset is a curated subset of PDBsum 23 , consisting of 2293 protein structures with 660 apo and 1633 holo structures.…”