Deep Learning for Detection of Pulmonary Metastasis on Chest Radiographs

Abstract

A rtificial intelligence using deep learning (DL) technology has recently exhibited promising performance in various fields of medicine, particularly medical image analyses (1-6). These initial successes have elevated expectations for implementing high-performance artificial intelligence in daily clinical practice to innovate and improve its quality and effectiveness (7,8).However, the introduction of DL in daily practice remains in its infancy. One of the most important challenges in bringing DL into daily practice is validating the performance of this technology in actual clinical practice (9,10). The vast majority of previous investigations evaluated the performance of DL systems using retrospectively and conveniently collected test data sets in an experimental manner (11). High performance in these investigations may be biased and cannot guarantee reproduction in realworld practice (9). Therefore, a DL system needs to be evaluated while being used in the actual clinical practice to understand its realistic and unbiased performance.Lung nodule identification in patients with cancer is a relevant task because it may indicate metastasis (9,12). Although chest CT is the standard examination for pulmonary metastasis evaluation, chest radiography may help in the surveillance for newly occurring metastasis at follow-up evaluations, especially in facilities with limited access to CT. However, radiologists' performance in detection of lung nodules on chest radiographs has been unsatisfactory (13-16). Therefore, the computer-aided Background: A computer-aided detection (CAD) system may help surveillance for pulmonary metastasis at chest radiography in situations where there is limited access to CT.Purpose: To evaluate whether a deep learning (DL)-based CAD system can improve diagnostic yield for newly visible lung metastasis on chest radiographs in patients with cancer. Materials and Methods:A regulatory-approved CAD system for lung nodules was implemented to interpret chest radiographs from patients referred by the medical oncology department in clinical practice. In this retrospective diagnostic cohort study, chest radiographs interpreted with assistance from a CAD system after the implementation (January to April 2019, CAD-assisted interpretation group) and those interpreted before the implementation (September to December 2018, conventional interpretation group) of the CAD system were consecutively included. The diagnostic yield (frequency of true-positive detections) and false-referral rate (frequency of false-positive detections) of formal reports of chest radiographs for newly visible lung metastasis were compared between the two groups using generalized estimating equations. Propensity score matching was performed between the two groups for age, sex, and primary cancer.Results: A total of 2916 chest radiographs from 1521 patients (1546 men, 1370 women; mean age, 62 years) and 5681 chest radiographs from 3456 patients (2941 men, 2740 women; mean age, 62 years) were analyzed in the CAD-assisted interpretation and...

Use of “Diagnostic Yield” in Imaging Research Reports: Results from Articles Published in Two General Radiology Journals

“…Table 1 and Figure 3 summarize the characteristics of these articles. Briefly, 64.7% (11 of 17) of the articles adopted “diagnostic yield” [ 45 47 48 49 53 64 65 67 68 72 73 ], and 35.3% (6 of 17) adopted “detection rate” [ 50 63 66 69 71 77 ]. Companion parameters for describing false-positive results were reported in 12 articles (70.6%, or 12 of 17) [ 47 49 50 53 64 65 66 67 68 71 73 77 ].…”

“…Briefly, 64.7% (11 of 17) of the articles adopted “diagnostic yield” [ 45 47 48 49 53 64 65 67 68 72 73 ], and 35.3% (6 of 17) adopted “detection rate” [ 50 63 66 69 71 77 ]. Companion parameters for describing false-positive results were reported in 12 articles (70.6%, or 12 of 17) [ 47 49 50 53 64 65 66 67 68 71 73 77 ]. Fifteen articles were diagnostic cohort studies [ 81 ] and two were clinical trials with cohort sizes ranging between 86 and 29138 patients (median: 524).…”

“…Hwang et al [ 64 ] and Kim et al [ 65 ] studies were good examples of using “diagnostic yield” and “false referral rate”. Additionally, Hwang et al [ 64 ] compared the diagnostic yield of computer-aided detection (CAD)-assisted chest radiography for detecting lung metastasis with that of conventional chest radiography in a cohort of 1521 outpatients. Diagnostic yield was calculated as the “number of radiographs with true-positive results/total number of radiographs,” and false referral rate was calculated as the “number of radiographs with false-positive results/total number of radiographs.” They demonstrated an improved diagnostic yield of CAD-assisted interpretation without increasing the false referral rate, thereby demonstrating the added value of CAD during patient care.…”

“…The proportion of studies reporting companion parameters was similar between the journals (72.7% vs. 66.7%). The false referral rate was used in only four articles (23.5%, 4 of 17) [ 64 65 66 68 ]. Improvements in using the terms correctly were observed with time in the studies published in Radiology (33.3% [3 of 9 articles from 2012–2016] vs. 80.0% [8 of 10 from 2017–2022]).…”

“…“Diagnostic yield” conveys a more clinically meaningful concept rather than simply presenting the performance of a diagnostic test represented by “sensitivity” and “specificity.” For example, if screening test A showed a higher diagnostic yield than screening test B, one can interpret that the screening test may further improve patient outcomes by detecting more diseases in a given study cohort. Hwang et al [ 64 ] provided a good example where an additional benefit of the CAD system was shown in interpreting pulmonary metastasis surveillance by demonstrating an increase in diagnostic yield compared to visual interpretation alone (0.86% vs. 0.32%). The other article also focused on the added value of the index test to that of the conventional screening test [ 66 ].…”