Estimating Error Rates in Bioactivity Databases

Tiikkainen, Pekka; Bellis, Louisa J.; Light, Yvonne; Franke, Lutz

doi:10.1021/ci400099q

Cited by 59 publications

(56 citation statements)

References 13 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In parallel to the concern about the evaluation of individual bioactivity predictions, recent publications have aimed at establishing the level of uncertainty in public bioactivity databases [22-25]. In this vein, Brown et al [26] highlighted the importance of including the uncertainty of bioactivity data into the evaluation of models quality.…”

Section: Introductionmentioning

confidence: 99%

Proteochemometric modeling in a Bayesian framework

et al. 2014

View full text Add to dashboard Cite

Proteochemometrics (PCM) is an approach for bioactivity predictive modeling which models the relationship between protein and chemical information. Gaussian Processes (GP), based on Bayesian inference, provide the most objective estimation of the uncertainty of the predictions, thus permitting the evaluation of the applicability domain (AD) of the model. Furthermore, the experimental error on bioactivity measurements can be used as input for this probabilistic model.In this study, we apply GP implemented with a panel of kernels on three various (and multispecies) PCM datasets. The first dataset consisted of information from 8 human and rat adenosine receptors with 10,999 small molecule ligands and their binding affinity. The second consisted of the catalytic activity of four dengue virus NS3 proteases on 56 small peptides. Finally, we have gathered bioactivity information of small molecule ligands on 91 aminergic GPCRs from 9 different species, leading to a dataset of 24,593 datapoints with a matrix completeness of only 2.43%.GP models trained on these datasets are statistically sound, at the same level of statistical significance as Support Vector Machines (SVM), with R02 values on the external dataset ranging from 0.68 to 0.92, and RMSEP values close to the experimental error. Furthermore, the best GP models obtained with the normalized polynomial and radial kernels provide intervals of confidence for the predictions in agreement with the cumulative Gaussian distribution. GP models were also interpreted on the basis of individual targets and of ligand descriptors. In the dengue dataset, the model interpretation in terms of the amino-acid positions in the tetra-peptide ligands gave biologically meaningful results.

show abstract

Section: Introductionmentioning

confidence: 99%

Proteochemometric modeling in a Bayesian framework

et al. 2014

View full text Add to dashboard Cite

show abstract

“…The aforementioned values are the result of machine-based harmonization and consolidation of multiple data objects in chemical, bioactivity and CCP space. An independent study by Tiikkainen and Franke (19), comparing ChEMBL (release 14) and WOMBAT 2012.01, showed >394 000 unique bioactivities in WOMBAT, compared with nearly 3.3 million bioactivities in ChEMBL; and 2755 unique targets in ChEMBL, compared with 1486 unique targets in WOMBAT. The harmonization trends suggest that a consolidated database is preferable to a federated collection, at least in this case, when seeking to evaluate global bioactivity trends.…”

Section: Discussionmentioning

confidence: 99%

“…The harmonization trends suggest that a consolidated database is preferable to a federated collection, at least in this case, when seeking to evaluate global bioactivity trends. This solution was, for example, implemented in the ‘Merz Virtual Bioactivity Database’, which integrates ChEMBL and WOMBAT, among other data sources (8, 19). …”

Section: Discussionmentioning

confidence: 99%

“…PDSP, IUPHAR-DB, ChEMBL and WOMBAT), we found that identical articles are covered by these resources, yet data are not always identical. Indeed, up to 3% errors in target protein identity, up to 2.7% errors in bioactivity values, and up to 7% errors in chemical structure depiction were found in comparing three data sources (19). In CARLSBAD, these tuples were harmonized by providing median values wherever possible, and by representing ‘higher curation’ values where possible, when multiple conflicting values were found.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

The CARLSBAD Database: A Confederated Database of Chemical Bioactivities

et al. 2013

View full text Add to dashboard Cite

Many bioactivity databases offer information regarding the biological activity of small molecules on protein targets. Information in these databases is often hard to resolve with certainty because of subsetting different data in a variety of formats; use of different bioactivity metrics; use of different identifiers for chemicals and proteins; and having to access different query interfaces, respectively. Given the multitude of data sources, interfaces and standards, it is challenging to gather relevant facts and make appropriate connections and decisions regarding chemical–protein associations. The CARLSBAD database has been developed as an integrated resource, focused on high-quality subsets from several bioactivity databases, which are aggregated and presented in a uniform manner, suitable for the study of the relationships between small molecules and targets. In contrast to data collection resources, CARLSBAD provides a single normalized activity value of a given type for each unique chemical–protein target pair. Two types of scaffold perception methods have been implemented and are available for datamining: HierS (hierarchical scaffolds) and MCES (maximum common edge subgraph). The 2012 release of CARLSBAD contains 439 985 unique chemical structures, mapped onto 1,420 889 unique bioactivities, and annotated with 277 140 HierS scaffolds and 54 135 MCES chemical patterns, respectively. Of the 890 323 unique structure–target pairs curated in CARLSBAD, 13.95% are aggregated from multiple structure–target values: 94 975 are aggregated from two bioactivities, 14 544 from three, 7 930 from four and 2214 have five bioactivities, respectively. CARLSBAD captures bioactivities and tags for 1435 unique chemical structures of active pharmaceutical ingredients (i.e. ‘drugs’). CARLSBAD processing resulted in a net 17.3% data reduction for chemicals, 34.3% reduction for bioactivities, 23% reduction for HierS and 25% reduction for MCES, respectively. The CARLSBAD database supports a knowledge mining system that provides non-specialists with novel integrative ways of exploring chemical biology space to facilitate knowledge mining in drug discovery and repurposing.Database URL: http://carlsbad.health.unm.edu/carlsbad/.

show abstract

“…Several studies notified that the bioactivity databases should be used with caution since errors in the data they provided could arise 62,86 (Table 2). A manual curation of all data is then required before inclusion to limit the integration of errors.…”

Section: Selection Of Active Compoundsmentioning

confidence: 99%

Benchmarking Data Sets for the Evaluation of Virtual Ligand Screening Methods: Review and Perspectives

Lagarde

Zagury

Montès

2015

J. Chem. Inf. Model.

View full text Add to dashboard Cite

Virtual screening methods are commonly used nowadays in drug discovery processes. However, to ensure their reliability, they have to be carefully evaluated. The evaluation of these methods is often realized in a retrospective way, notably by studying the enrichment of benchmarking data sets. To this purpose, numerous benchmarking data sets were developed over the years, and the resulting improvements led to the availability of high quality benchmarking data sets. However, some points still have to be considered in the selection of the active compounds, decoys, and protein structures to obtain optimal benchmarking data sets.

show abstract

Estimating Error Rates in Bioactivity Databases

Cited by 59 publications

References 13 publications

Proteochemometric modeling in a Bayesian framework

Proteochemometric modeling in a Bayesian framework

The CARLSBAD Database: A Confederated Database of Chemical Bioactivities

Benchmarking Data Sets for the Evaluation of Virtual Ligand Screening Methods: Review and Perspectives

Contact Info

Product

Resources

About