Analysis of CASP8 targets, predictions and assessment methods

Abstract

Results of the recent Critical Assessment of Techniques for Protein Structure Prediction, CASP8, present several valuable sources of information. First, CASP targets comprise a realistic sample of currently solved protein structures and exemplify the corresponding challenges for predictors. Second, the plethora of predictions by all possible methods provides an unusually rich material for evolutionary analysis of target proteins. Third, CASP results show the current state of the field and highlight specific problems in both predicting and assessing. Finally, these data can serve as grounds to develop and analyze methods for assessing prediction quality. Here we present results of our analysis in these areas. Our objective is not to duplicate CASP assessment, but to use our unique experience as former CASP5 assessors and CASP8 predictors to (i) offer more insights into CASP targets and predictions based on expert analysis, including invaluable analysis prior to target structure release; and (ii) develop an assessment methodology tailored towards current challenges in the field. Specifically, we discuss preparing target structures for assessment, parsing protein domains, balancing evaluations based on domains and on whole chains, dividing targets into categories and developing new evaluation scores. We also present evolutionary analysis of the most interesting and challenging targets.Database URL: Our results are available as a comprehensive database of targets and predictions at http://prodata.swmed.edu/CASP8.

An ambiguity principle for assigning protein structural domains

“…3H). Our algorithm also finds the same three-domain organization as CATH and the four-domain assignment made by N. Grishin ( 15 )—and therefore annotated as a manual assignment in ECOD. Although SWORD does not find the domain assignment that is most favorable for predicting this target structure, the multiplicity of partitioning solutions it provides remarkably reflects the complex evolutionary history of this protein.…”

“…Therefore, extending the fold library with alternative domain decompositions for each template structure can improve the search for compatible folds and, consequently, the quality of protein structure predictions. This can be verified by using target structures from the eighth edition of the Critical Assessment of Structure Prediction (CASP8) competition, for which the modeling difficulty is lowered when treating the structural domains separately rather than the whole protein chain ( 15 ), the challenge being reduced to the relative positioning of the individually modeled domains. For these target proteins, unlike annotations from other methods, SWORD partitioning finds the number of domains that most facilitates structure prediction (Fig.…”

“…In detail, the Rieske ferredoxin (target T0391) is annotated as a one-domain protein by PDP, CATH, and Pfam (no data in SCOP), although this target is an obvious two–evolutionary domain protein, constituted of a rubredoxin-like domain (residues 55 to 117) inserted into a six-strand β barrel (residues 14 to 54 and 118 to 154), as shown by N. Grishin in his analysis of the CASP8 results ( 15 ) and as can be found in his ECOD database. This two-domain partitioning corresponds to what SWORD proposes as the best alternative decomposition (Fig.…”

On the Assessment of Structural Protein Models with ROSETTA-Design and HMMer: Value, Potential and Limitations

“…Again, this problem can be tackled with a number of strategies, and most of them have been implemented as computer software programs, and their validity tested at the CASP contests (Shi et al, 2009). …”

“…None of them achieves 100% success, and even the most successful can fail where other, usually less reliable, may succeed (Kryshtafovych et al, 2009;Melo & Feytmans, 1998 This last kind are usually designated as unrefined models. c. Unrefined models can be recognized with various energy scoring strategies (Luthy et al, 1992;Shi et al, 2009;Hu & Jiang, 2010;Melo & Feytmans, 1998); and can be corrected through the use of molecular mechanics software (Rosales-León et al, 2012;Fiser & Sali 2003), though this approach has limitations, as mentioned before (Faver et al, 2011;Hu & Jiang, 2010;Melo & Feytmans, 19985). d. Wrong models instead may frequently be deceitful, because, due to their systematic error [5], a molecular mechanics force-field may report a low energy value, as long as the chemical and geometrical details are well refined.…”