Improving the Performance of Radiologists Using Artificial Intelligence-Based Detection Support Software for Mammography: A Multi-Reader Study

Abstract

Objective To evaluate whether artificial intelligence (AI) for detecting breast cancer on mammography can improve the performance and time efficiency of radiologists reading mammograms. Materials and Methods A commercial deep learning-based software for mammography was validated using external data collected from 200 patients, 100 each with and without breast cancer (40 with benign lesions and 60 without lesions) from one hospital. Ten readers, including five breast specialist radiologists (BSRs) and five general radiologists (GRs), assessed all mammography images using a seven-point scale to rate the likelihood of malignancy in two sessions, with and without the aid of the AI-based software, and the reading time was automatically recorded using a web-based reporting system. Two reading sessions were conducted with a two-month washout period in between. Differences in the area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, and reading time between reading with and without AI were analyzed, accounting for data clustering by readers when indicated. Results The AUROC of the AI alone, BSR (average across five readers), and GR (average across five readers) groups was 0.915 (95% confidence interval, 0.876–0.954), 0.813 (0.756–0.870), and 0.684 (0.616–0.752), respectively. With AI assistance, the AUROC significantly increased to 0.884 (0.840–0.928) and 0.833 (0.779–0.887) in the BSR and GR groups, respectively ( p = 0.007 and p < 0.001, respectively). Sensitivity was improved by AI assistance in both groups (74.6% vs. 88.6% in BSR, p < 0.001; 52.1% vs. 79.4% in GR, p < 0.001), but the specificity did not differ significantly (66.6% vs. 66.4% in BSR, p = 0.238; 70.8% vs. 70.0% in GR, p = 0.689). The average reading time pooled across readers was significantly decreased by AI assistance for BSRs (82.73 vs. 73.04 seconds, p < 0.001) but increased in GRs (35.44 vs. 42.52 seconds, p < 0.001). Conclusion AI-based software improved the performance of radiologists regardless of their experience and affected the reading time.

Survey on Value Elements Provided by Artificial Intelligence and Their Eligibility for Insurance Coverage With an Emphasis on Patient-Centered Outcomes

“…The supplementary literature search identified 48 eligible studies ( Fig. 1 ) [ 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 ]. Table 6 shows the count of studies that addressed the four value elements provided by AI.…”

The impact of artificial intelligence on the reading times of radiologists for chest radiographs

“…For mammography, studies have shown conflicting results, with reading times not being significantly affected by the use of AI 16 or decreasing up to 22.3% when AI results are available 17 . In a study by Lee et al, reading times were affected by the experience levels of radiologists even with AI, as general radiologists showed longer reading times; breast radiologists did not show any change in reading times with AI use 8 . Interestingly, a study by Pacile et al reported results for mammography that were similar to the findings seen in this study 18 .…”

“…Recent studies have demonstrated better diagnostic performance with AI when reprioritizing brain computed tomography (CT) for the detection of hemorrhage 6 , 7 . Integration of AI into mammography has been found to enhance the diagnostic performance of radiologists without increasing reading time 8 . A similar tendency was observed in the detection of bone fractures using radiographs 9 , 10 .…”

Positive Predictive Values of Abnormality Scores From a Commercial Artificial Intelligence-Based Computer-Aided Diagnosis for Mammography

“…The sensitivity varies from 60%–90% and is significantly affected by breast density [ 1 2 ]. Recently, artificial intelligence-based computer-aided diagnosis (AI-CAD) has been increasingly integrated into mammography, displaying a diagnostic accuracy comparable to or even superior to that of radiologists, while significantly enhancing the diagnostic performance of radiologists [ 3 4 5 6 7 ].…”