A Targeted Review of Breast Cancer Studies of Concordance for an Internationally-Implemented Artificially Intelligent Clinical Decision-Support System

Abstract

Background: Breast cancer has the highest incidence and is the leading cause of cancer-related mortality among women worldwide. IBM Watson® for Oncology (WfO), an artificial intelligence-based clinical decision-support system, provides therapeutic options for consideration to cancer-treating physicians. We conducted a targeted review of studies evaluating concordance of therapeutic options offered by the system with treatment decisions by practicing clinicians in breast cancer. Methods: PubMed, EMBASE, Cochrane, trial registers, conference abstracts, and an internal publication database were searched to identify studies evaluating the concordance of system-generated therapeutic options with treatment decisions by individual clinicians and multidisciplinary tumor boards for breast cancer patients reported in peer-reviewed abstracts or papers published in English between 01/01/2015 and 11/15/2019.Results: Ten breast cancer concordance studies (4703 patients) that met the inclusion criteria were identified and analyzed; the identified studies were from China, India, and Thailand. The weighted mean concordance for all studies was 67.4% (SD 16.0%, range 55.0% - 98.0%). The weighted mean concordance of the system with multidisciplinary tumor boards was 88.2%, (SD 9.7%, range 76.5% - 98.0%), which was substantially higher than concordance between the system and individual clinicians (61.5% , SD 10.1%, range 55.0% -76.0%).Conclusion: Concordance between system-generated therapeutic options and treatment decisions of multidisciplinary tumor boards or individual clinicians for breast cancer demonstrated overall agreement between the system and decisions of practicing cancer-treating physicians in China, India and Thailand. As multidisciplinary tumor boards may lead to higher quality clinical decision-making compared to those of individual clinicians in practice, the relatively higher concordance of the system with multidisciplinary tumor boards suggests a role for clinical decision support to inform clinicians of evidence-informed treatment options.