Validation of deep learning-based computer-aided detection software use for interpretation of pulmonary abnormalities on chest radiographs and examination of factors that influence readers’ performance and final diagnosis

Abstract

Purpose To evaluate the performance of a deep learning-based computer-aided detection (CAD) software for detecting pulmonary nodules, masses, and consolidation on chest radiographs (CRs) and to examine the effect of readers’ experience and data characteristics on the sensitivity and final diagnosis. Materials and methods The CRs of 453 patients were retrospectively selected from two institutions. Among these CRs, 60 images with abnormal findings (pulmonary nodules, masses, and consolidation) and 140 without abnormal findings were randomly selected for sequential observer-performance testing. In the test, 12 readers (three radiologists, three pulmonologists, three non-pulmonology physicians, and three junior residents) interpreted 200 images with and without CAD, and the findings were compared. Weighted alternative free-response receiver operating characteristic (wAFROC) figure of merit (FOM) was used to analyze observer performance. The lesions that readers initially missed but CAD detected were stratified by anatomic location and degree of subtlety, and the adoption rate was calculated. Fisher’s exact test was used for comparison. Results The mean wAFROC FOM score of the 12 readers significantly improved from 0.746 to 0.810 with software assistance (P = 0.007). In the reader group with < 6 years of experience, the mean FOM score significantly improved from 0.680 to 0.779 (P = 0.011), while that in the reader group with ≥ 6 years of experience increased from 0.811 to 0.841 (P = 0.12). The sensitivity of the CAD software and the adoption rate for the lesions with subtlety level 2 or 3 (obscure) lesions were significantly lower than for level 4 or 5 (distinct) lesions (50% vs. 93%, P < 0.001; and 55% vs. 74%, P = 0.04, respectively). Conclusion CAD software use improved doctors’ performance in detecting nodules/masses and consolidation on CRs, particularly for non-expert doctors, by preventing doctors from missing distinct lesions rather than helping them to detect obscure lesions.

New trend in artificial intelligence-based assistive technology for thoracic imaging

“…Recent studies have shown that AI has high performance in detecting pulmonary nodules on chest radiographs and CT scans [ 36 – 41 ]. In the detection studies reviewed for chest radiography using the AI-based computer-aided detection/diagnosis (CAD) system [ 37 , 39 , 40 ], the sensitivity ranges from 79.0 to 91.1% and the specificity from 93 to 100%.…”

The impact of large language models on radiology: a guide for radiologists on the latest innovations in AI

“…This represents a foundational shift in automated feature extraction from imaging data, consequently reducing the time and expertise required for interpreting medical images. Additionally, DL-powered tools have demonstrated their efficacy in improving diagnostic accuracy by aiding radiologists in precisely detecting anomalies such as tumors, external injuries, and other pathological conditions [16][17][18][19][20]. These advancements not only accelerate the diagnostic process but also contribute substantially to prognostic evaluations, thus playing a crucial role in elevating patient care and outcomes [21].…”

Fairness of artificial intelligence in healthcare: review and recommendations

“…One important reason for this is the global shortage of radiologists [ 18 – 21 ]. In particular, Japan has numerous publications on AI use in the field of radiology, including X-ray [ 22 – 25 ], mammography [ 26 – 30 ], US [ 31 ], CT [ 32 – 40 ], MRI [ 41 – 50 ], and PET [ 51 , 52 ]. This surge in Radiological AI publications in Japan could be related to Japan having both the lowest number of radiologists per capita and the highest number of CT and MRI machines per capita among the Organization for Economic Co-operation and Development (OECD) countries [ 53 ].…”