The Added Value of a Computer‐Aided Diagnosis System in Differential Diagnosis of Breast Lesions by Radiologists With Different Experience

Abstract

Objectives-To evaluate the value of the computer-aided diagnosis system, S-Detect (based on deep learning algorithm), in distinguishing benign and malignant breast masses and reducing unnecessary biopsy based on the experience of radiologists.Methods-From February 2018 to March 2019, 266 breast masses in 192 women were included in our study. Ultrasound (US) examination, including S-Detect technique, was performed by the radiologist with about 10 years of clinical experience in breast US imaging. US images were analyzed by four other radiologists with different experience in breast imaging (radiologists 1, 2, 3, and 4 with 1, 4, 9, and 20 years, respectively) according to their clinical experience (with and without the results of S-Detect). Diagnostic capabilities and unnecessary biopsy of radiologists and radiologists combined with S-Detect were compared and analyzed.Results-After referring to the results of S-Detect, the changes made by less experienced radiologists were greater than experienced radiologists (benign or malignant, 44 vs 22 vs 14 vs 2; unnecessary biopsy, 34 vs 25 vs 10 vs 5). When combined with S-Detect, less experienced radiologists showed significant improvement in accuracy, specificity, positive predictive value, negative predictive value, and area under curve (P < .05), but not for experienced radiologists (P > .05). Similarly, the unnecessary biopsy rate of less experienced radiologists decreased significantly (44.4% vs 32.7%, P = .006; 36.8% vs 28.2%, P = .033), but not for experienced radiologists (P > .05).Conclusions-Less experienced radiologists rely more on S-Detect software. And S-Detect can be an effective decision-making tool for breast US, especially for less experienced radiologists.

The added value of S-detect in the diagnostic accuracy of breast masses by senior and junior radiologist groups: a systematic review and meta-analysis

“…We extracted and summarized the basic characteristics of the included literature. Four included studies were prospective ( 20 , 21 , 23 , 26 ), seven were retrospective ( 18 , 19 , 25 , 29 - 31 , 33 ), and the remaining articles were not reported. Also, a total of 3,349 patients with 3,895 lesions were included.…”

“…It was suggested that the diagnostic performance was high in both groups; pooled sensitivity, the pooled specificity and AUC were all higher in the senior radiologist group than in the junior radiologist group, and the AUC value was significantly different between the two groups (P<0.01). Some articles showed that the improvement was more significant in the junior radiologist group compared to US (19)(20)(21)23). Meanwhile, we analyzed the clinical application value of US + S-detect by plotting Fagan diagrams, setting the pre-test probability at 50% and the post-test probability at 87% and 81% for the senior and junior groups, respectively.…”

“…Combining S-detect can also reduce the biopsy rate by downgrading BI-RADS 4a masses. Combining S-detect to diagnose breast masses reduced the rate of unnecessary biopsies ( 22 , 23 , 25 ), but downgrading can also result in false-negative results for malignant nodes, so further training in S-detect and the empirical judgment of radiologists is also important.…”

“…This meta-analysis included 19 original studies (18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32)(33)(34)(35)(36), including eight English articles (19)(20)(21)(22)(23)(24)(25)(26) and eleven Chinese studies (18,(27)(28)(29)(30)(31)(32)(33)(34)(35)(36). We extracted and summarized the basic characteristics of the included literature.…”

How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)

“…For example, in medical diagnostics the evaluation of system accuracy, and accordingly its trustworthiness, may require a comparison with how well human experts typically perform in the same task. Whereas a novice user will perform significantly worse without the system, an expert user might perform better without the system [113]. Consequently, if a user assesses the trustworthiness of a system by referring to the individual standard: "Will the system provide better outcomes than I would?…”