Concordance Study Between IBM Watson for Oncology and Clinical Practice for Patients with Cancer in China

Abstract

IBM Watson for Oncology (WFO) has begun to be used in China. In this study, concordance was examined between the treatment recommendation proposed by WFO and clinical decisions for 362 patients in our cancer center, which could reflect the differences of cancer treatment between China and the U.S. Different cancer types showed different concordances, and only gastric cancers were significantly less likely to be concordant. Incidence and pharmaceuticals may be the major causes of discordance. To be comprehensively and rapidly applied in China, WFO needs to accelerate localization. This study may have a significant effect on application of artificial intelligence systems in China.

Concordance in postsurgical radioactive iodine therapy recommendations between Watson for Oncology and clinical practice in patients with differentiated thyroid carcinoma

“…This suggests that WFO recommendations may be based on the treatment preferences of a small number of physicians who provided the data, and not actual insights gained by analyzing actual patients. Previous studies of this subject demonstrated that the concordance rate varied by countries and cancer types . As mentioned earlier, a retrospective study of patients in India who were diagnosed with breast cancer demonstrated a treatment recommendation concordance rate of 93% between WFO and the multidisciplinary tumor board .…”

“…As mentioned earlier, a retrospective study of patients in India who were diagnosed with breast cancer demonstrated a treatment recommendation concordance rate of 93% between WFO and the multidisciplinary tumor board . However, another study of Chinese patients with breast cancer revealed a concordance rate of 64% . In addition, 2 studies of concordance for lung cancer treatment demonstrated rates of 66% and 81%, respectively, whereas rates for gastric cancer were found to be very low in 2 studies at only 12% and 49%, respectively …”

“…WFO reduces the ambiguity in optimal therapy decision making that can manifest when personal preferences or cognitive biases dominate individual decisions, thus paving the way for standardized treatments . To enable generalized screening, it is critical to achieve concordance in the recommendations rendered by WFO and those offered by physicians to verify the accuracy of the former . A double‐blind study from India found that breast cancer treatment recommendations made by both WFO and a multidisciplinary tumor board were highly concordant .…”

“…Among the supported cases, the rate of treatment concordance between WFO and the expert group was 66% . A recent study that encompassed patients from China diagnosed with several types of cancer demonstrated that the concordance between WFO's and physicians' decisions was not as high as previously reported in other countries; moreover, concordance rates varied by cancer type, ranging from 12% to 96% …”

“…1,4 To enable generalized screening, it is critical to achieve concordance in the recommendations rendered by WFO and those offered by physicians to verify the accuracy of the former. 1,2,5 A double-blind study from India found that breast cancer treatment recommendations made by both WFO and a multidisciplinary tumor board were highly concordant. 1 In another retrospective study of Chinese patients with lung cancer, WFO did not support approximately 18% of the cases.…”