Comparative analysis of methods for evaluation of protein models against native structures

Olechnovič, Kliment; Monastyrskyy, Bohdan; Kryshtafovych, Andriy; Venclovas, C̆eslovas

doi:10.1093/bioinformatics/bty760

Cited by 40 publications

(52 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This is the approach generally taken in CASP assessment, and the ability of the group to rank their models forms an implicit part of the ranking score. Any ranking score that assigns comparable weights to a combination of metrics measuring global fold, local fold, and estimated model accuracy is likely to lead to a similar overall ranking, as the metrics within these general categories tend to be highly correlated to one another . Because the ASE accuracy self‐estimate score measures an orthogonal characteristic of the models (and to assess the possibility that a good ASE score could be attained by assigning large errors to poor models), we also tested the effect on ranking of excluding the ASE measure.…”

Section: Resultssupporting

confidence: 80%

“…Over the years, a large number of evaluation measures have been developed to assess different aspects of model quality. A detailed description, classification and review of a number of these metrics has been published recently; they differ for instance in whether or not they depend on structure superposition and whether they depend on global or local measures. Most of these metrics are computed, collated, and analyzed by the Prediction Center (http://predictioncenter.org), making them much more convenient for assessors and others.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of template‐based modeling in CASP13

et al. 2019

Self Cite

View full text Add to dashboard Cite

Performance in the template-based modeling (TBM) category of CASP13 is assessed here, using a variety of metrics. Performance of the predictor groups that participated is ranked using the primary ranking score that was developed by the assessors for CASP12. This reveals that the best results are obtained by groups that include contact predictions or inter-residue distance predictions derived from deep multiple sequence alignments. In cases where there is a good homologue in the wwPDB (TBM-easy category), the best results are obtained by modifying a template. However, for cases with poorer homologues (TBM-hard), very good results can be obtained without using an explicit template, by deep learning algorithms trained on the wwPDB. Alternative metrics are introduced, to allow testing of aspects of structural models that are not addressed by traditional CASP metrics. These include comparisons to the main-chain and side-chain torsion angles of the target, and the utility of models for solving crystal structures by the molecular replacement method. The alternative metrics are poorly correlated with the traditional metrics, and it is proposed that modeling has reached a sufficient level of maturity that the best models should be expected to satisfy this wider range of criteria.

show abstract

Section: Resultssupporting

confidence: 80%

Section: Methodsmentioning

confidence: 99%

Evaluation of template‐based modeling in CASP13

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…To assess accuracy of crosslinking‐assisted models and their improvement over the corresponding non‐assisted predictions, we employed the GDT_TS measure for monomeric predictions, and the LDDT measure for multimeric ones. Comparative analysis of these measures is provided in a recently published paper …”

Section: Methodsmentioning

confidence: 99%

Assessment of chemical‐crosslink‐assisted protein structure modeling in CASP13

et al. 2019

Self Cite

View full text Add to dashboard Cite

With the advance of experimental procedures obtaining chemical crosslinking information is becoming a fast and routine practice. Information on crosslinks can greatly enhance the accuracy of protein structure modeling. Here, we review the current state of the art in modeling protein structures with the assistance of experimentally determined chemical crosslinks within the framework of the 13th meeting of Critical Assessment of Structure Prediction approaches. This largest‐to‐date blind assessment reveals benefits of using data assistance in difficult to model protein structure prediction cases. However, in a broader context, it also suggests that with the unprecedented advance in accuracy to predict contacts in recent years, experimental crosslinks will be useful only if their specificity and accuracy further improved and they are better integrated into computational workflows.

show abstract

“…In general, the VoroMQA‐based grouping is fairly similar to that based on CAD‐score. This is quite remarkable considering that lDDT and CAD‐score are some of the most similarly behaved and highly correlated reference‐based scores …”

Section: Discussionmentioning

confidence: 85%

Estimation of model accuracy in CASP13

Choe²,

et al. 2019

Self Cite

View full text Add to dashboard Cite

Methods to reliably estimate the accuracy of 3D models of proteins are both a fundamental part of most protein folding pipelines and important for reliable identification of the best models when multiple pipelines are used. Here, we describe the progress made from CASP12 to CASP13 in the field of estimation of model accuracy (EMA) as seen from the progress of the most successful methods in CASP13. We show small but clear progress, that is, several methods perform better than the best methods from CASP12 when tested on CASP13 EMA targets. Some progress is driven by applying deep learning and residue‐residue contacts to model accuracy prediction. We show that the best EMA methods select better models than the best servers in CASP13, but that there exists a great potential to improve this further. Also, according to the evaluation criteria based on local similarities, such as lDDT and CAD, it is now clear that single model accuracy methods perform relatively better than consensus‐based methods.

show abstract

Comparative analysis of methods for evaluation of protein models against native structures

Abstract: Supplementary data are available at Bioinformatics online.

Cited by 40 publications

References 25 publications

Evaluation of template‐based modeling in CASP13

Evaluation of template‐based modeling in CASP13

Assessment of chemical‐crosslink‐assisted protein structure modeling in CASP13

Estimation of model accuracy in CASP13

Contact Info

Product

Resources

About