Performance of in silico tools for the evaluation of p16INK4a (CDKN2A) variants in CAGI

Abstract

Correct phenotypic interpretation of variants of unknown significance for cancer-associated genes is a diagnostic challenge as genetic screenings gain in popularity in the next-generation sequencing era. The Critical Assessment of Genome Interpretation (CAGI) experiment aims to test and define the state of the art of genotype-phenotype interpretation. Here, we present the assessment of the CAGI p16INK4a challenge. Participants were asked to predict the effect on cellular proliferation of ten variants for the p16INK4a tumor suppressor, a cyclin-dependent kinase inhibitor encoded by the CDKN2A gene. Twenty-two pathogenicity predictors were assessed with a variety of accuracy measures for reliability in a medical context. Different assessment measures were combined in an overall ranking to provide more robust results. The R scripts used for assessment are publicly available from a GitHub repository for future use in similar assessment exercises. Despite a limited test-set size, our findings show a variety of results, with some methods performing significantly better. Methods combining different strategies frequently outperform simpler approaches. The best predictor, Yang&Zhou lab, uses a machine learning method combining an empirical energy function measuring protein stability with an evolutionary conservation term. The p16INK4a challenge highlights how subtle structural effects can neutralize otherwise deleterious variants.

Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations

“…Other predictors integrate additional features including biophysical properties of amino acids, protein functional annotations and epigenetic data (8). Protein structural information, derived from experimentally determined models, is also used by several methods (9,10), although there is conflicting information over whether its inclusion significantly improves predictor performance (11).…”

Reports from CAGI: The Critical Assessment of Genome Interpretation

“…CAGI challenges span a wide range of relationships between genetic variation and disease. For single base variants, there are challenges that address the problem of interpreting the impact of missense mutations on protein activity using a variety of molecular and cellular phenotypes, challenges that test the ability to predict the effect of mutations in cancer driver genes on cell growth, and challenges on the effect of single‐base variants on RNA expression levels and splicing (including Beer, ; Capriotti, Martelli, Fariselli, & Casadio, ; Carraro et al., ; Katsonis & Lichtarge, ; Kreimer et al., ; Niroula & Vihinen ; Pejaver et al., ; Tang et al., 2017; Tang & Fenton, ; Xu et al., ; Yin et al., ; Zeng, Edwards, Guo, & Gifford, ; Zhang et al., ). At the level of full exome and genome sequence, there are challenges that assess methods for assigning complex traits phenotypes and that evaluate the ability to associate genome sequence and an extensive profile of phenotypic traits (including Cai et al., 2017; Daneshjou et al., ; Daneshjou et al., ; Giollo et al., ; Laksshman, Bhat, Viswanath, & Li, ; Pal, Kundu, Yin, & Moult, ; Wang et al., ).…”

Performance of computational methods for the evaluation of pericentriolar material 1 missense variants in CAGI‐5

“…Submissions were compared with experimental data to evaluate their prediction performance. Using a set of performance measures highlighting the strengths and weaknesses of each predictor similar to previous CAGI assessments (Carraro et al, ).…”