Evaluation of in silico tools for variant classification in clinically actionable NSCLC variants.

Abstract

e13545 Background: Genetic variants beyond FDA-approved drug targets are often identified in NSCLC patients. To address this challenge, in silico variant classification tools are available to determine whether specific variants contribute to disease pathogenicity or remain benign. Although the performance of in silico tools has been analyzed in previous studies, it has not been analyzed for actionable targets of FDA-approved therapies for NSCLC. The aim of this study is to compare the performance of commonly used in silico tools in classifying the pathogenicity of actionable variants in NSCLC. Methods: We evaluated the performance of several in silico tools: PolyPhen-2, Align-GVGD, and MutationTaster2. A curated set of targetable NSCLC missense variants (n = 179) was used. The dataset consisted of variants in the BRAF, EGFR, ERBB2, KRAS, MET, ALK, and ROS1 genes based on their indications as molecular targets in the NCCN Guidelines for NSCLC. Pathogenic variants (n = 80) were curated based on available literature and annotations according to the NCCN Guidelines, OncoKB, My Cancer Genome, and AACR Project GENIE. Benign variants (n = 99) were curated from the dbSNP database with the inclusion criteria of a benign or likely benign ClinVar assertion. The overall accuracy, sensitivity, specificity, and Matthews correlation coefficient (MCC) of each in silico tool were determined. The performance of each in silico tool in predicting pathogenicity for subsets of sensitizing (n = 18) and resistant (n = 57) variants was also evaluated. Results: PolyPhen-2 HumVar demonstrated the highest overall accuracy (0.80), specificity (0.69), and MCC (0.63) of the in silico tools analyzed. PolyPhen-2 HumDiv (0.75) and MutationTaster2 (0.69) had similar overall accuracies while Align-GVGD (0.50) had the lowest overall accuracy. Align-GVGD also had the lowest MCC (0.08), with the other in silico tools ranging from 0.50 to 0.63. All the in silico tools achieved high sensitivities, with MutationTaster2 performing the highest (1.00) and Align-GVGD performing the lowest (0.86). The specificities were remarkably low (0.20-0.69) for all the in silico tools, with the lowest specificity demonstrated by Align-GVGD. The overall accuracies when classifying the subsets of sensitizing and resistant variants were generally high, ranging from 0.84 to 1.00. Conclusions: These results suggest that the performance of the evaluated in silico tools to predict the pathogenicity of clinically actionable NSCLC missense variants is not fully reliable. The tools analyzed in this study could be acceptable to rule out pathogenicity in variants given their higher sensitivities, but are limited when it comes to identifying pathogenicity in variants due to low specificities.

Evaluation ofin silico toolsfor variant classification in missense variants of solid cancer with actionable genetic targets

“…Align-GVGD's worst overall performance in this study aligns with a previous study investigating in silico tools for variant classification in clinically actionable NSCLC variants [17]. Moreover, Align-GVGD resulted in an MCC greater than zero and less than 0.2 in all cancer types, which suggests that the model's performance is better than random chance but is still significantly limited in its predictive ability.…”

“…Our selection of 9 common solid cancers included breast, prostate, colorectal, melanoma of skin, bladder/urothelial, pancreatic, thyroid, ovarian, and biliary cancers. Lung cancer was not included because the accuracy of in silico predictors for variants related to NSCLC has been analyzed in a previous study [17].…”

“…In the present study, we evaluated six in silico tools: Polyphen-2 (consisting of two models: HumDiv, HumVar), Align-GVGD, MutationTaster2021, FATHMM, CADD, and REVEL [8][9][10][11][12][13]. These commonly-used tools were selected based on their inclusion in the ACMG Standards and Guidelines [4], as well as further literature [17].…”