Assessment of methods for predicting the effects of PTEN and TPMT protein variants

Abstract

Thermodynamic stability is a fundamental property shared by all proteins. Changes in stability due to mutation are a widespread molecular mechanism in genetic diseases. Methods for the prediction of mutation‐induced stability change have typically been developed and evaluated on incomplete and/or biased data sets. As part of the Critical Assessment of Genome Interpretation, we explored the utility of high‐throughput variant stability profiling (VSP) assay data as an alternative for the assessment of computational methods and evaluated state‐of‐the‐art predictors against over 7,000 nonsynonymous variants from two proteins. We found that predictions were modestly correlated with actual experimental values. Predictors fared better when evaluated as classifiers of extreme stability effects. While different methods emerging as top performers depending on the metric, it is nontrivial to draw conclusions on their adoption or improvement. Our analyses revealed that only 16% of all variants in VSP assays could be confidently defined as stability‐affecting. Furthermore, it is unclear as to what extent VSP abundance scores were reasonable proxies for the stability‐related quantities that participating methods were designed to predict. Overall, our observations underscore the need for clearly defined objectives when developing and using both computational and experimental methods in the context of measuring variant impact.

Variant effect predictions capture some aspects of deep mutational scanning experiments

“…Both SIFT [39] and PolyPhen-2 [37] are optimized for capturing binary effects, not correlations, as confirmed by recent studies [47,49]. Consequently, analysis for these was confined to binary predictions.…”

“…DMS datasets constitute a uniquely valuable resource for the evaluation of current SAV effect prediction methods [17,47,48], not the least, because most have not used those data. The Fowler lab has, recently, published an excellent analysis of prediction methods on DMS datasets and developed a new regression-based prediction method, Envision, trained only on DMS data [49].…”

Reports from the fifth edition of CAGI: The Critical Assessment of Genome Interpretation

“…Over all CAGI editions, the plurality of challenges have been on the interpretation of isolated missense variants, and CAGI5 continues that trend. There are assessment, data provider, and participant papers for the prediction of the destabilizing effect of missense mutations in a cancer‐relevant protein (Frataxin, with biophysical measurements of protein stability; Petrosino et al, ; Savojardo, Petrosino et al, ; Strokach, Corbi‐Verge, & Kim, ); on the effect of missense changes in a human calmodulin, assayed using a high‐throughput yeast complementation assay (Zhang et al, ); the effect of missense mutations related to schizophrenia in human Pericentriolar Material 1 ( PCM1 ), using a zebrafish development model (Miller, Wang, & Bromberg, ; Monzon et al, ); the effect of missense mutations in two cancer‐related proteins, PTEN and TPMT , on intracellular protein levels, measured in a high‐throughput assay (Pejaver et al, ); and the effect of missense changes in a monogenic disease related protein, acid alpha‐glucosidase ( GAA ), with measurements of total intracellular enzyme activity (Adhikari, ). Three participant papers describe results on all the missense challenges (Garg & Pal, ; Katsonis & Lichtarge, ; Savojardo, Babbi et al, ).…”

funtrp: identifying protein positions for variation driven functional tuning

“…To test our model, we collected three additional DMS datasets, covering the PTEN, TPMT and HSP90 proteins (47,48) which were NOT used in training. Note, that for HSP90 the knockout variant effect measures were not directly available; we thus approximated knockout scores as the mean of the unnormalized effect scores (0.15) reported for variants in eight critical positions, i.e.…”