Distribution of evaluation scores for the models submitted to the second cryo-EM model challenge

Kryshtafovych, Andriy; Monastyrskyy, Bohdan; Adams, Paul D.; Lawson, Catherine L.; Chiu, Wah

doi:10.1016/j.dib.2018.08.214

Cited by 5 publications

(7 citation statements)

References 18 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CASP infrastructure for running the evaluation, reporting scores, and visualizing evaluation results for cryo‐EM targets (http://predictioncenter.org/casp13/cryoem_results.cgi) is based on the prototype of the evaluation infrastructure developed for the cryo‐EM model challenge …”

Section: Methodsmentioning

confidence: 99%

Cryo‐electron microscopy targets in CASP13: Overview and evaluation of results

et al. 2019

Self Cite

View full text Add to dashboard Cite

Structures of seven CASP13 targets were determined using cryo‐electron microscopy (cryo‐EM) technique with resolution between 3.0 and 4.0 Å. We provide an overview of the experimentally derived structures and describe results of the numerical evaluation of the submitted models. The evaluation is carried out by comparing coordinates of models to those of reference structures (CASP‐style evaluation), as well as checking goodness‐of‐fit of modeled structures to the cryo‐EM density maps. The performance of contributing research groups in the CASP‐style evaluation is measured in terms of backbone accuracy, all‐atom local geometry and similarity of inter‐subunit interfaces. The results on the cryo‐EM targets are compared with those on the whole set of eighty CASP13 targets. A posteriori refinement of the best models in their corresponding cryo‐EM density maps resulted in structures that are very close to the reference structure, including some regions with better fit to the density.

show abstract

Section: Methodsmentioning

confidence: 99%

Cryo‐electron microscopy targets in CASP13: Overview and evaluation of results

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…(Kryshtafovych et al, 2018a) and also accessible from the main Results web page through the score distributions link http://model-compare.emdatabank.org/em_score_boxplots.cgi ). The box plots clearly show that score distributions on models built starting from reference models versus ab initio are very different.…”

Section: Brief Analysis Of the Resultsmentioning

confidence: 99%

“…This highlights some of the challenges assessing ab initio models, which are often incomplete in structure and/or sequence. Figures 4 and 5 in (Kryshtafovych et al, 2018a) show distribution of evaluation scores when all models are grouped together (i.e., without splitting them into ab initio and optimization categories). Outliers in the graphs for the complete set of models are almost all ab-initio models.…”

Section: Brief Analysis Of the Resultsmentioning

confidence: 99%

Evaluation system and web infrastructure for the second cryo-EM model challenge

Kryshtafovych

Adams

Lawson

et al. 2018

Journal of Structural Biology

Self Cite

View full text Add to dashboard Cite

An evaluation system and a web infrastructure were developed for the second cryo-EM model challenge. The evaluation system includes tools to validate stereo-chemical plausibility of submitted models, check their fit to the corresponding density maps, estimate their overall and per-residue accuracy, and assess their similarity to reference cryo-EM or X-ray structures as well as other models submitted in this challenge. The web infrastructure provides a convenient interface for analyzing models at different levels of detail. It includes interactively sortable tables of evaluation scores for different subsets of models and different sublevels of structure organization, and a suite of visualization tools facilitating model analysis. The results are publicly accessible at http://model-compare.emdatabank.org.

show abstract

“…In parallel, a second group was charged with evaluating whether improved models could be built from several recently published reconstructions. Interestingly, many of the submitted models were better than the published ones ( 65 , 66 ). During these two challenge activities, many participants were stimulated to improve data processing algorithms, modeling pipelines, and to generate novel validation procedures; outcomes included the need in particular for careful review of metrics that evaluate the fit of the model to its cryo-EM map ( 63 ).…”

Section: Accelerated Activities In Cryo-em Data Archiving and Structure Validation Since 2010mentioning

confidence: 90%

Evolution of standardization and dissemination of cryo-EM structures and data jointly by the community, PDB, and EMDB

Chiu

Schmid

Pintilie

et al. 2021

Journal of Biological Chemistry

Self Cite

View full text Add to dashboard Cite

Cryogenic electron microscopy (cryo-EM) methods began to be used in the mid-1970s to study thin and periodic arrays of proteins. Following a half-century of development in cryo-specimen preparation, instrumentation, data collection, data processing, and modeling software, cryo-EM has become a routine method for solving structures from large biological assemblies to small biomolecules at near to true atomic resolution. This review explores the critical roles played by the Protein Data Bank (PDB) and Electron Microscopy Data Bank (EMDB) in partnership with the community to develop the necessary infrastructure to archive cryo-EM maps and associated models. Public access to cryo-EM structure data has in turn facilitated better understanding of structure–function relationships and advancement of image processing and modeling tool development. The partnership between the global cryo-EM community and PDB and EMDB leadership has synergistically shaped the standards for metadata, one-stop deposition of maps and models, and validation metrics to assess the quality of cryo-EM structures. The advent of cryo-electron tomography (cryo-ET) for in situ molecular cell structures at a broad resolution range and their correlations with other imaging data introduce new data archival challenges in terms of data size and complexity in the years to come.

show abstract

Distribution of evaluation scores for the models submitted to the second cryo-EM model challenge

Abstract: 142 protein structure models were submitted to second Cryo-EM model challenge (2015–2016). Accuracy of the models was evaluated with 54 evaluation scores. Results of the descriptive statistical analysis of the scores are provided in this article.

Cited by 5 publications

References 18 publications

Cryo‐electron microscopy targets in CASP13: Overview and evaluation of results

Cryo‐electron microscopy targets in CASP13: Overview and evaluation of results

Evaluation system and web infrastructure for the second cryo-EM model challenge

Evolution of standardization and dissemination of cryo-EM structures and data jointly by the community, PDB, and EMDB

Contact Info

Product

Resources

About