A novel workflow for the safe and effective integration of AI as supporting reader in double reading breast cancer screening: A large-scale retrospective evaluation

Abstract

Objectives To evaluate the effectiveness of a novel strategy for using AI as a supporting reader for the detection of breast cancer in mammography-based double reading screening practice. Instead of replacing a human reader, here AI serves as the second reader only if it agrees with the recall/no-recall decision of the first human reader. Otherwise, a second human reader makes an assessment, enacting standard human double reading. Design Retrospective large-scale, multi-site, multi-device, evaluation study. Participants 280,594 cases from 180,542 female participants who were screened for breast cancer with digital mammography between 2009 and 2019 at seven screening sites in two countries (UK and Hungary). Main outcome measures Primary outcome measures were cancer detection rate, recall rate, sensitivity, specificity, and positive predictive value. Secondary outcome was reduction in workload measured as arbitration rate and number of cases requiring second human reading. Results The novel workflow was found to be superior or non-inferior on all screening metrics, almost halving arbitration and reducing the number of cases requiring second human reading by up to 87.50% compared to human double reading. Conclusions AI as a supporting reader adds a safety net in case of AI discordance compared to alternative workflows where AI replaces the second human reader. In the simulation using large-scale historical data, the proposed workflow retains screening performance of the standard of care of human double reading while drastically reducing the workload. Further research should study the impact of the change in case mix for the second human reader as they would only assess cases where the AI and first human reader disagree.