The second round of Critical Assessment of Automated Structure Determination of Proteins by NMR: CASD-NMR-2013

Abstract

The second round of the community-wide initiative Critical Assessment of automated Structure Determination of Proteins by NMR (CASD-NMR-2013) comprised ten blind target datasets, consisting of unprocessed spectral data, assigned chemical shift lists and unassigned NOESY peak and RDC lists, that were made available in both curated (i.e. manually refined) or un-curated (i.e. automatically generated) form. Ten structure calculation programs, using fully automated protocols only, generated a total of 164 three-dimensional structures (entries) for the ten targets, sometimes using both curated and un-curated lists to generate multiple entries for a single target. The accuracy of the entries could be established by comparing them to the corresponding manually solved structure of each target, which was not available at the time the data were provided. Across the entire data set, 71 % of all entries submitted achieved an accuracy relative to the reference NMR structure better than 1.5 Å. Methods based on NOESY peak lists achieved even better results with up to 100 % of the entries within the 1.5 Å threshold for some programs. However, some methods did not converge for some targets using un-curated NOESY peak lists. Over 90 % of the entries achieved an accuracy better than the more relaxed threshold of 2.5 Å that was used in the previous CASD-NMR-2010 round. Comparisons between entries generated with un-curated versus curated peaks show only marginal improvements for the latter in those cases where both calculations converged.Electronic supplementary materialThe online version of this article (doi:10.1007/s10858-015-9953-4) contains supplementary material, which is available to authorized users.

Protein structure prediction assisted with sparse NMR data in CASP13

“…In general, models with reasonable accuracy were (as judged by MolProbity score) physically reasonable structures, although some models with good Molprobity scores were not particularly accurate. Inaccurate models with good MolProbity scores have also been observed in assessments of incorrect homology models and of inaccurate CASD‐NMR experimental NMR structures …”

“…The ideal input for NMR‐assisted prediction would be the unassigned NOESY peak lists together with sequence‐specific NMR assignments and RDC data, as was done in the CASD‐NMR project . However, in order to reduce the extent of domain specific NMR spectroscopy knowledge required for participation in CASP13, the organizers instead provided these NOESY data as “Ambiguous Contact Lists” (Supplementary Figure S2).…”

“…NMR structures were modeled using the ASDP program for NOESY peak assignment, together with Cyana for structure generation from the resulting restraints. This pipeline, described in detail elsewhere, was one of the top performing automated NOESY analysis methods in the CASD‐NMR experiments . ASDP uses expert system methods to assign NOESY cross peaks, and to generate distance restraints.…”

Probabilistic validation of protein NMR chemical shift assignments

“…In addition, for proteins with structures deposited into the Protein Data Bank (PDB) (Berman et al 2007), we compared ARECA’s performance against three validation packages that rely on 3D structures: VASCO (Rieping and Vranken 2010, Vranken and Rieping 2009), SHIFTCOR (Zhang et al 2003), and SPARTA+ (Shen and Bax 2007, Shen and Bax 2010). These comparisons were conducted on a set of 16 proteins that included 10 target proteins from the second round of CASD-NMR (Rosato et al 2015, Wassenaar et al 2012) and 6 other proteins from the Biological Magnetic Resonance Data Bank (BMRB) (Ulrich et al 2008). The other validation methods rely on chemical shift statistics as the basis for flagging outlier atoms that could have valid or incorrect assignments.…”

Automated protein structure determination by NMR

“…In addition to the contributions specifically dedicated to the achievement of each software, which altogether provide an extensive view of where automation in NMR structure determination stands, two articles focus respectively on the performance achieved for the targets included in the second round of CASD-NMR and on the validation of the structures submitted by the CASD-NMR participants. The experimental NMR data for all 20 targets in the two rounds of CASD-NMR are publicly available (see Rosato et al 2015 for details).…”