Comprehensive Evaluation of Protein Structure Alignment Methods: Scoring by Geometric Measures

Abstract

We report the largest and most comprehensive comparison of protein structural alignment methods. Specifically, we evaluate six publicly available structure alignment programs: SSAP, STRUCTAL, DALI, LSQMAN, CE and SSM by aligning all 8,581,970 protein structure pairs in a test set of 2930 protein domains specially selected from CATH v.2.4 to ensure sequence diversity.We consider an alignment good if it matches many residues, and the two substructures are geometrically similar. Even with this definition, evaluating structural alignment methods is not straightforward. At first, we compared the rates of true and false positives using receiver operating characteristic (ROC) curves with the CATH classification taken as a gold standard. This proved unsatisfactory in that the quality of the alignments is not taken into account: sometimes a method that finds less good alignments scores better than a method that finds better alignments. We correct this intrinsic limitation by using four different geometric match measures (SI, MI, SAS, and GSAS) to evaluate the quality of each structural alignment. With this improved analysis we show that there is a wide variation in the performance of different methods; the main reason for this is that it can be difficult to find a good structural alignment between two proteins even when such an alignment exists.We find that STRUCTAL and SSM perform best, followed by LSQMAN and CE. Our focus on the intrinsic quality of each alignment allows us to propose a new method, called "Best-of-All" that combines the best results of all methods. Many commonly used methods miss 10-50% of the good Best-of-All alignments.By putting existing structural alignments into proper perspective, our study allows better comparison of protein structures. By highlighting limitations of existing methods, it will spur the further development of better structural alignment methods. This will have significant biological implications now that structural comparison has come to play a central role in the analysis of experimental work on protein structure, protein function and protein evolution.

The structure of human ADP-ribosylhydrolase 3 (ARH3) provides insights into the reversibility of protein ADP-ribosylation

“…The mitochondrial targeting signal sequence was predicted with TargetP (40). Structure models were superimposed with LSQMAN (56). Ribbon plots and hydrogen-bond presentations were prepared with MOLSCRIPT and Raster3D (57,58) and surface presentations with GRASP (33) and PyMOL (44,59).…”

The limits of protein sequence comparison?

“…Recently, several authors have suggested that the accuracy of structure alignments, rather than search sensitivity, is a more useful measure of the effectiveness of sequence [20,21] and structure [22] comparison methods. Sequence similarity scores are calculated from implicit alignments; however, sensitivity -the ability to assign statistically significant similarity scores to distant homologs -is distinct from alignment accuracy.…”

Growth of novel protein structural data

“…24). One could also match known structures using structural alignment (25). Parsing the chains into structural domains could be done as described above for sequence matching and augmented with automated domain finding programs (26).…”

“…All chains that are highly similar in sequence or structure can be considered to contribute redundant structural information to the database. Here, we focus on sequence similarity which is easier to calculate and can also be applied more widely than structural comparison (25). Sequence similarity is measured by comparing all 44,374 nonidentical chain sequences with one another using Pearson's FASTA program (40).…”