CAGI5: Objective performance assessments of predictions based on the Evolutionary Action equation

Abstract

Many computational approaches estimate the effect of coding variants, but their predictions often disagree with each other. These contradictions confound users and raise questions regarding reliability. Performance assessments can indicate the expected accuracy for each method and highlight advantages and limitations. The Critical Assessment of Genome Interpretation (CAGI) community aims to organize objective and systematic assessments: They challenge predictors on unpublished experimental and clinical data and assign independent assessors to evaluate the submissions. We participated in CAGI experiments as predictors, using the Evolutionary Action (EA) method to estimate the fitness effect of coding mutations. EA is untrained, uses homology information, and relies on a formal equation: The fitness effect equals the functional sensitivity to residue changes multiplied by the magnitude of the substitution. In previous CAGI experiments (between 2011 and 2016), our submissions aimed to predict the protein activity of single mutants. In 2018 (CAGI5), we also submitted predictions regarding clinical associations, folding stability, and matching genomic data with phenotype. For all these diverse challenges, we used EA to predict the fitness effect of variants, adjusted to specifically address each question. Our submissions had consistently good performance, suggesting that EA predicts reliably the effects of genetic variants.

Assessment of blind predictions of the clinical significance of BRCA1 and BRCA2 variants

“…We have also indicated which methods were executed as published; for the others, further details are available in the Supporting Information section. The Lictarge Lab submitted predictions with Evolutionary Action ( EA ; special issue; Katsonis & Lichtarge, ). EA estimates variant pathogenicity through evolutionary information by using an analytic equation.…”

“…The Lictarge Lab submitted predictions with Evolutionary Action ( EA ; special issue; Katsonis & Lichtarge, ). EA estimates variant pathogenicity through evolutionary information by using an analytic equation.…”

“…• The Lictarge Lab submitted predictions with Evolutionary Action (EA; special issue; Katsonis & Lichtarge, 2019 Normalized EA Normalizes EA predictions with the estimated fraction of BRCA isoforms affected by the mutation.…”

Reports from the fifth edition of CAGI: The Critical Assessment of Genome Interpretation

“…There are assessment, data provider, and participant papers for the prediction of the destabilizing effect of missense mutations in a cancer‐relevant protein (Frataxin, with biophysical measurements of protein stability; Petrosino et al, ; Savojardo, Petrosino et al, ; Strokach, Corbi‐Verge, & Kim, ); on the effect of missense changes in a human calmodulin, assayed using a high‐throughput yeast complementation assay (Zhang et al, ); the effect of missense mutations related to schizophrenia in human Pericentriolar Material 1 ( PCM1 ), using a zebrafish development model (Miller, Wang, & Bromberg, ; Monzon et al, ); the effect of missense mutations in two cancer‐related proteins, PTEN and TPMT , on intracellular protein levels, measured in a high‐throughput assay (Pejaver et al, ); and the effect of missense changes in a monogenic disease related protein, acid alpha‐glucosidase ( GAA ), with measurements of total intracellular enzyme activity (Adhikari, ). Three participant papers describe results on all the missense challenges (Garg & Pal, ; Katsonis & Lichtarge, ; Savojardo, Babbi et al, ). The issue also contains assessment articles from two earlier missense challenges on monogenic disease related proteins: N‐acetyl‐glucosaminidase ( NAGLU; Clark et al, ), with total intracellular enzyme activity measured; and cystathionine beta‐synthase ( CBS ), using the metric of yeast growth in a complication assay (Kasak, Bakolitsa et al, ).…”

Evolutionary Action Score ofTP53Mutations: Integrated Clinico-pathologic And Protein Structural Analysis in Myelodysplastic Syndromes

“…The EAp53 scoring system ranges between 0 to 100, with a higher score indicating a higher impact on downstream transcription and protein function. EAp53 scoring system has been shown to be a reliable prognostic marker in patients with head and neck and colorectal cancers 25,26,34 and in objective assessments 35,36 .…”