Performance of an artificial intelligence-based annotation algorithm for reporting cancer genomic profiling tests.

Abstract

1551 Background: Cancer genomic profiling (CGP) tests have been approved in Japan since June 2019, with the requisite that all test results be discussed by molecular tumor boards (MTBs). More than 20,000 patients in over 200 designated hospitals have taken CGP tests by December 2021. As CGP tests have entered clinical practice, streamlining decision making by MTBs and standardizing interpretation of test results and treatment recommendations have become urgent issues. Here, we evaluated the utility of Chrovis, an annotation algorithm for reporting CGP tests to support MTBs make their recommendations. Methods: We retrospectively reviewed the reporting process of all approved CGP tests done at The University of Tokyo Hospital between December 2019 and November 2021. Chrovis provided annotation for each genetic variant by incorporating biologic, clinical, and therapeutic information by referencing several public knowledge databases and using natural language processing, and generated reports using the automated program. The MTB reviewed and made any necessary changes before finalizing the report. Changes in disclosure of germline findings were made according to the recommendations of a national guideline with consideration of past and family history. Results: Of the 243 tests, 91 changes in 81 Chrovis reports (33% of all reports) were made by the MTB. The most common type of change was germline disclosure with 26 changes (29%), followed by clinical trial information in Japan (18 changes, 20%) and recommendation of the patient-proposed national basket trial with multiple targeted agents (17 changes, 19%). Changes in germline disclosure increased from June 2021, when an update to a national guideline was released, while the proportion of changes in the latter two types remained unchanged. Gene alterations that led to the highest number of changes was TP53, with 13 changes. Changes in therapeutic recommendations were frequently observed in the RAS/MAPK pathway ( BRAF, KRAS, NF1, NRAS) with 12 changes. More changes were required with a tumor-only tissue CGP panel (57 of 149) compared with a matched tumor/normal tissue CGP panel (24 of 94, p = 0.04), mostly due to germline disclosure (24 vs. 2 changes). Conclusions: We observed that automated algorithm-based reporting was sufficient in 67% of reports. Recommendation for germline disclosure still requires manual supervision, particularly with tumor-only tissue CGP panels if algorithms do not incorporate medical history. The process of recommending clinical trials needs improvement, e.g., standardizing database formats for inclusion and exclusion criteria.

Human resources for administrative work to carry out a comprehensive genomic profiling test in Japan

“…Artificial intelligence‐assisted annotation for each genetic variant can help finalize Expert Panel reports by incorporating biologic, clinical, and therapeutic information through public knowledge databases. However, Expert Panels made changes in 33% of all AI‐assisted reports and the most common type of change was germline disclosure, which accounted for 29% 28 . Although a common criteria for evaluation of germline findings is broadly applied in each hospital in Japan, 29 AI‐assisted annotation alone might not be sufficient for simplification of the Expert Panels.…”

“…However, Expert Panels made changes in 33% of all AI‐assisted reports and the most common type of change was germline disclosure, which accounted for 29%. 28 Although a common criteria for evaluation of germline findings is broadly applied in each hospital in Japan, 29 AI‐assisted annotation alone might not be sufficient for simplification of the Expert Panels.…”

“…28 Although a common criteria for evaluation of germline findings is broadly applied in each hospital in Japan,29 AI-assisted annotation alone might not be sufficient for simplification of the…”