Frequency of Missed Findings on Chest Radiographs (CXRs) in an International, Multicenter Study: Application of AI to Reduce Missed Findings

Kaviani, Parisa; Kalra, Mannudeep K.; Digumarthy, Subba R.; Gupta, Reya V.; Dasegowda, Giridhar; Jagirdar, Ammar; Gupta, Salil; Putha, Preetham; Mahajan, Vidur; Reddy, Bhargava; Venugopal, Vasanth K.; Tadepalli, Manoj; Bizzo, Bernardo C.; Dreyer, Keith J.

doi:10.3390/diagnostics12102382

Cited by 6 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the conservative operating point, the AI system still missed four urgent findings, and one of these misses was a CTconfirmed hilar mass. However, it is assumable that radiologists would have a comparable miss rate, as described in the literature [21,[25][26][27][28]. This is also reflected in the performance of the original report in the study by Plesner et al, in which the radiologist reached a sensitivity of 93.5% for 'critical' abnormalities.…”

Section: Discussionmentioning

confidence: 93%

Performance of AI to exclude normal chest radiographs to reduce radiologists’ workload

Schalekamp,

van Leeuwen,

Calli

et al. 2024

Eur Radiol

View full text Add to dashboard Cite

Introduction This study investigates the performance of a commercially available artificial intelligence (AI) system to identify normal chest radiographs and its potential to reduce radiologist workload. Methods Retrospective analysis included consecutive chest radiographs from two medical centers between Oct 1, 2016 and Oct 14, 2016. Exclusions comprised follow-up exams within the inclusion period, bedside radiographs, incomplete images, imported radiographs, and pediatric radiographs. Three chest radiologists categorized findings into normal, clinically irrelevant, clinically relevant, urgent, and critical. A commercial AI system processed all radiographs, scoring 10 chest abnormalities on a 0–100 confidence scale. AI system performance was evaluated using the area under the ROC curve (AUC), assessing the detection of normal radiographs. Sensitivity was calculated for the default and a conservative operating point. the detection of negative predictive value (NPV) for urgent and critical findings, as well as the potential workload reduction, was calculated. Results A total of 2603 radiographs were acquired in 2141 unique patients. Post-exclusion, 1670 radiographs were analyzed. Categories included 479 normal, 332 clinically irrelevant, 339 clinically relevant, 501 urgent, and 19 critical findings. The AI system achieved an AUC of 0.92. Sensitivity for normal radiographs was 92% at default and 53% at the conservative operating point. At the conservative operating point, NPV was 98% for urgent and critical findings, and could result in a 15% workload reduction. Conclusion A commercially available AI system effectively identifies normal chest radiographs and holds the potential to lessen radiologists’ workload by omitting half of the normal exams from reporting. Clinical relevance statement The AI system is able to detect half of all normal chest radiographs at a clinically acceptable operating point, thereby potentially reducing the workload for the radiologists by 15%. Key Points The AI system reached an AUC of 0.92 for the detection of normal chest radiographs. Fifty-three percent of normal chest radiographs were identified with a NPV of 98% for urgent findings. AI can reduce the workload of chest radiography reporting by 15%.

show abstract

Section: Discussionmentioning

confidence: 93%

Performance of AI to exclude normal chest radiographs to reduce radiologists’ workload

Schalekamp,

van Leeuwen,

Calli

et al. 2024

Eur Radiol

View full text Add to dashboard Cite

show abstract

“…CXR is often the first imaging modality for the detection of lung disease and, thus, shapes disease management strategy ( 1 ). Despite the significance and obvious advantages, the inconsistent interpretation of CXR and associated reporting delays continue to be a significant burden in various healthcare settings ( 2 – 5 ). Alternatively, recent advances in convolutional neural networks have led to the birth of several stand-alone artificial intelligence (AI) tools to interpret CXR.…”

mentioning

confidence: 99%

Implementation of Artificial Intelligence–Assisted Chest X-ray Interpretation: It Is About Time

Ram

Bodduluri

2023

Annals ATS

View full text Add to dashboard Cite

Artificial intelligence-assisted double reading of chest radiographs to detect clinically relevant missed findings: a two-centre evaluation

Topff,

Steltenpool,

Ranschaert

et al. 2024

Eur Radiol

View full text Add to dashboard Cite

Objectives To evaluate an artificial intelligence (AI)–assisted double reading system for detecting clinically relevant missed findings on routinely reported chest radiographs. Methods A retrospective study was performed in two institutions, a secondary care hospital and tertiary referral oncology centre. Commercially available AI software performed a comparative analysis of chest radiographs and radiologists’ authorised reports using a deep learning and natural language processing algorithm, respectively. The AI-detected discrepant findings between images and reports were assessed for clinical relevance by an external radiologist, as part of the commercial service provided by the AI vendor. The selected missed findings were subsequently returned to the institution’s radiologist for final review. Results In total, 25,104 chest radiographs of 21,039 patients (mean age 61.1 years ± 16.2 [SD]; 10,436 men) were included. The AI software detected discrepancies between imaging and reports in 21.1% (5289 of 25,104). After review by the external radiologist, 0.9% (47 of 5289) of cases were deemed to contain clinically relevant missed findings. The institution’s radiologists confirmed 35 of 47 missed findings (74.5%) as clinically relevant (0.1% of all cases). Missed findings consisted of lung nodules (71.4%, 25 of 35), pneumothoraces (17.1%, 6 of 35) and consolidations (11.4%, 4 of 35). Conclusion The AI-assisted double reading system was able to identify missed findings on chest radiographs after report authorisation. The approach required an external radiologist to review the AI-detected discrepancies. The number of clinically relevant missed findings by radiologists was very low. Clinical relevance statement The AI-assisted double reader workflow was shown to detect diagnostic errors and could be applied as a quality assurance tool. Although clinically relevant missed findings were rare, there is potential impact given the common use of chest radiography. Key Points • A commercially available double reading system supported by artificial intelligence was evaluated to detect reporting errors in chest radiographs (n=25,104) from two institutions. • Clinically relevant missed findings were found in 0.1% of chest radiographs and consisted of unreported lung nodules, pneumothoraces and consolidations. • Applying AI software as a secondary reader after report authorisation can assist in reducing diagnostic errors without interrupting the radiologist’s reading workflow. However, the number of AI-detected discrepancies was considerable and required review by a radiologist to assess their relevance. Graphical Abstract

show abstract

Frequency of Missed Findings on Chest Radiographs (CXRs) in an International, Multicenter Study: Application of AI to Reduce Missed Findings

Cited by 6 publications

References 26 publications

Performance of AI to exclude normal chest radiographs to reduce radiologists’ workload

Performance of AI to exclude normal chest radiographs to reduce radiologists’ workload

Implementation of Artificial Intelligence–Assisted Chest X-ray Interpretation: It Is About Time

Artificial intelligence-assisted double reading of chest radiographs to detect clinically relevant missed findings: a two-centre evaluation

Contact Info

Product

Resources

About