Critical assessment of high-throughput standalone methods for secondary structure prediction

Abstract

Sequence-based prediction of protein secondary structure (SS) enjoys wide-spread and increasing use for the analysis and prediction of numerous structural and functional characteristics of proteins. The lack of a recent comprehensive and large-scale comparison of the numerous prediction methods results in an often arbitrary selection of a SS predictor. To address this void, we compare and analyze 12 popular, standalone and high-throughput predictors on a large set of 1975 proteins to provide in-depth, novel and practical insights. We show that there is no universally best predictor and thus detailed comparative studies are needed to support informed selection of SS predictors for a given application. Our study shows that the three-state accuracy (Q3) and segment overlap (SOV3) of the SS prediction currently reach 82% and 81%, respectively. We demonstrate that carefully designed consensus-based predictors improve the Q3 by additional 2% and that homology modeling-based methods are significantly better by 1.5% Q3 than ab initio approaches. Our empirical analysis reveals that solvent exposed and flexible coils are predicted with a higher quality than the buried and rigid coils, while inverse is true for the strands and helices. We also show that longer helices are easier to predict, which is in contrast to longer strands that are harder to find. The current methods confuse 1-6% of strand residues with helical residues and vice versa and they perform poorly for residues in the β- bridge and 3(10)-helix conformations. Finally, we compare predictions of the standalone implementations of four well-performing methods with their corresponding web servers.

Structural Protein Descriptors in 1-Dimension and their Sequence-Based Predictions

“…These mistakes are less prevalent in the tertiary structure predictions where the corresponding rates are at about 0.6%. Overall, we note that there is no clear-cut winner, i.e., none of the methods obtains favorable prediction quality on all measures; a similar conclusion was drawn in a recent evaluation of standalone secondary structure predictors [171]. Although PORTER_H obtains the highest overall Q 3 and SOV 3 , the runner-up SSpro, SPINE and YASPIN provide relatively high-quality predictions for helices and coils, coils, and strands, respectively.…”

SPINE X: Improving protein secondary structure prediction by multistep learning coupled with prediction of solvent accessible surface area and backbone torsion angles

“…Clearly, the definition of secondary structure, i.e., the methods for making secondary structure assignment, will have a direct impact on the accuracy of secondary structure prediction. The discrepancy among different automatic assignment techniques, as large as 15–25% 6, 7 , and inconsistency among assigned secondary structures within a single method 7 are among the reasons for the slow progress in improving secondary structure prediction in recent years 1, 8, 4, 9 . A recent critical assessment 9 suggests that the three-state accuracy for the best ab-initio single methods is around 80.5% based on a benchmark of 1975 proteins uploaded to the PDB 10 between 2004 and 2008.…”

From local structure to a global framework: recognition of protein folds

“…The exposed coils were predicted higher than helices and strands. The accuracies of prediction for residues in 3 10 helices and b-bridges were less than 50% and 48%, respectively [138]. Complementing profile -profile comparisons with predicted secondary structure resulted in a significant improvement in the efficiency of fold recognition [95,108,139,140].…”

“…A recent assessment of available secondary structure prediction methods suggests that a prediction accuracy of about 82% could be reached [138]. The exposed coils were predicted higher than helices and strands.…”