Artificial Intelligence Approach for Variant Reporting

Abstract

Purpose Next-generation sequencing technologies are actively applied in clinical oncology. Bioinformatics pipeline analysis is an integral part of this process; however, humans cannot yet realize the full potential of the highly complex pipeline output. As a result, the decision to include a variant in the final report during routine clinical sign-out remains challenging. Methods We used an artificial intelligence approach to capture the collective clinical sign-out experience of six board-certified molecular pathologists to build and validate a decision support tool for variant reporting. We extracted all reviewed and reported variants from our clinical database and tested several machine learning models. We used 10-fold cross-validation for our variant call prediction model, which derives a contiguous prediction score from 0 to 1 (no to yes) for clinical reporting. Results For each of the 19,594 initial training variants, our pipeline generates approximately 500 features, which results in a matrix of > 9 million data points. From a comparison of naive Bayes, decision trees, random forests, and logistic regression models, we selected models that allow human interpretability of the prediction score. The logistic regression model demonstrated 1% false negativity and 2% false positivity. The final models’ Youden indices were 0.87 and 0.77 for screening and confirmatory cutoffs, respectively. Retraining on a new assay and performance assessment in 16,123 independent variants validated our approach (Youden index, 0.93). We also derived individual pathologist-centric models (virtual consensus conference function), and a visual drill-down functionality allows assessment of how underlying features contributed to a particular score or decision branch for clinical implementation. Conclusion Our decision support tool for variant reporting is a practically relevant artificial intelligence approach to harness the next-generation sequencing bioinformatics pipeline output when the complexity of data interpretation exceeds human capabilities.

Clinically Integrated Molecular Diagnostics in Adenoid Cystic Carcinoma

“…Fusions for MYB‐NFIB or MYBL1‐NFIB (40%) and NOTCH1 mutations (25%) were the most common alterations. Across all cancer types tested in our institution (n = 2,701 cases were genotyped between 2013 and 2018) , we have only seen MYB and MYBL1 fusions in adenoid cystic carcinomas (specificity: 100%) . We modeled that even if a non‐ACC would harbor one MYB/MYBL1 fusion, it would take ∼14 cases of non‐ACC for the specificity to drop below 99.5%.…”

“…genotyped between 2013 and 2018) [27,37], we have only seen MYB and MYBL1 fusions in adenoid cystic carcinomas (specificity: 100%) [38]. We modeled that even if a non-ACC would harbor one MYB/MYBL1 fusion, it would take 14 cases of non-ACC for the specificity to drop below 99.5%.…”

Translating cancer genomics into precision medicine with artificial intelligence: applications, challenges and future perspectives

“…Massachusetts General Hospital (MGH) did the experiment and got very promising results. They selected ~ 500 features, built multiple ML models on ~ 20,000 clinical sign-out variants reported by board-certified molecular pathologists and then compared the prediction results to find the best model (Zomnir et al 2018 ). The logistic regression model demonstrated the best performance with only 1% false negativity and 2% false positivity, which is comparable to human decisions.…”

“…For instance, in one study, Watson for Genomics identified genomic alterations with potential clinical impact that were not recognized by the traditional molecular tumor boards in 323 (32%) of patients using an analysis that took only a few minutes (Patel et al 2018 ). At MGH, the clinical implementation of an AI-based decision support tool for variant reporting allows molecular pathologists to quickly make decisions and empowers them to explore the underlying reasoning behind them (Zomnir et al 2018 ). As we move to an age of AI, medical education must move beyond the foundational biomedical and clinical sciences to knowledge of information platforms and intelligence tools in healthcare and the skills to effectively use them (Wartman and Combs 2018 ).…”

“Somatic” and “pathogenic” - is the classification strategy applicable in times of large-scale sequencing?

“…The use of artificial intelligence in clinical oncology, genome interpretation, and especially in variant reporting has gained momentum. 49,50 Data available from manually classified variants can be used to train deep neural networks. 49 The advantage of this approach is that the algorithm is able to autonomously extract relevant features for classification and identify important combinations not only for genetic information but for all types of biomarker.…”