AI-based improvement in lung cancer detection on chest radiographs: results of a multi-reader study in NLST dataset

Yoo, Hyunsuk; Lee, Sang Hyup; Arru, Chiara; Khera, Ruhani Doda; Singh, Ramandeep; Siebert, Sean; Kim, Do-Hoon; Lee, Yuna; Park, Jae Young; Eom, Hye Joung; Digumarthy, Subba R.; Kalra, Mannudeep K.

doi:10.1007/s00330-021-08074-7

Cited by 48 publications

(31 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These results are compatible with other similar studies with improved reader performance with use of AI algorithms for detecting pulmonary nodules on chest radiographs. 2 Specifically, Yoo et al 12 evaluated 5485 images both with and without malignant pulmonary nodules from National Lung Cancer Screening Trial data and reported a greater improvement in sensitivity (8% with AI-aided interpretation using a different algorithm than the one used in our study, as compared with unaided interpretation) and a much smaller gain in specificity (2%). 12 Likewise, in another lung cancer screening data set, Jang et al 13 reported an improved detection of malignant pulmonary nodules with another AI algorithm to an AUC of 76% from 67% AUC with unaided interpretation.…”

Section: Discussionmentioning

confidence: 59%

“…Because of its accessibility, low cost, low radiation dose exposure, and reasonable diagnostic accuracy, chest radiographs are frequently used for early detection of pulmonary nodules despite their inferiority to low-dose computed tomography (CT). [1][2][3] Pulmonary nodules are common initial radiologic manifestations of lung cancer, but can be easily missed when they are subtle, small, or in difficult areas. [4][5][6][7] Detection of pulmonary nodules on chest radiographs has been a focus of several computeraided detection commercial and research software solutions for the past 2 decades 8,9 ; however, early solutions were limited due to their low sensitivity or high false positive rates.…”

Section: Introductionmentioning

confidence: 99%

“…[4][5][6][7] Detection of pulmonary nodules on chest radiographs has been a focus of several computeraided detection commercial and research software solutions for the past 2 decades 8,9 ; however, early solutions were limited due to their low sensitivity or high false positive rates. 9 Although several recent studies have examined how AI may improve the performance of radiologists in detection of pulmonary nodules on CXRs, 2,3 they are limited in their scope owing to the lack of multisite readers with different levels of clinical experience, detection difficulties, and settings. To address these limitations in the evaluation of a novel AI algorithm, we assessed if our AI algorithm can help detect pulmonary nodules on chest radiographs at different levels of detection difficulty with both normal and nonnormal control images in a multireader, multicenter setting.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An Artificial Intelligence–Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study

et al. 2021

Self Cite

View full text Add to dashboard Cite

IMPORTANCEMost early lung cancers present as pulmonary nodules on imaging, but these can be easily missed on chest radiographs. OBJECTIVE To assess if a novel artificial intelligence (AI) algorithm can help detect pulmonary nodules on radiographs at different levels of detection difficulty. DESIGN, SETTING, AND PARTICIPANTSThis diagnostic study included 100 posteroanterior chest radiograph images taken between 2000 and 2010 of adult patients from an ambulatory health care center in Germany and a lung image database in the US. Included images were selected to represent nodules with different levels of detection difficulties (from easy to difficult), and comprised both normal and nonnormal control.EXPOSURES All images were processed with a novel AI algorithm, the AI Rad Companion Chest X-ray. Two thoracic radiologists established the ground truth and 9 test radiologists from Germany and the US independently reviewed all images in 2 sessions (unaided and AI-aided mode) with at least a 1-month washout period. MAIN OUTCOMES AND MEASURESEach test radiologist recorded the presence of 5 findings (pulmonary nodules, atelectasis, consolidation, pneumothorax, and pleural effusion) and their level of confidence for detecting the individual finding on a scale of 1 to 10 (1 representing lowest confidence; 10, highest confidence). The analyzed metrics for nodules included sensitivity, specificity, accuracy, and receiver operating characteristics curve area under the curve (AUC). RESULTSImages from 100 patients were included, with a mean (SD) age of 55 (20) years and including 64 men and 36 women. Mean detection accuracy across the 9 radiologists improved by 6.4% (95% CI, 2.3% to 10.6%) with AI-aided interpretation compared with unaided interpretation. Partial AUCs within the effective interval range of 0 to 0.2 false positive rate improved by 5.6% (95% CI, −1.4% to 12.0%) with AI-aided interpretation. Junior radiologists saw greater improvement in sensitivity for nodule detection with AI-aided interpretation as compared with their senior counterparts (12%; 95% CI, 4% to 19% vs 9%; 95% CI, 1% to 17%) while senior radiologists experienced similar improvement in specificity (4%; 95% CI, −2% to 9%) as compared with junior radiologists (4%; 95% CI, −3% to 5%). CONCLUSIONS AND RELEVANCEIn this diagnostic study, an AI algorithm was associated with improved detection of pulmonary nodules on chest radiographs compared with unaided interpretation for different levels of detection difficulty and for readers with different experience.

show abstract

Section: Discussionmentioning

confidence: 59%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Artificial Intelligence–Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Further research is warranted to verify the efficacy of AI assistance in terms of patients' management or safety. For example, AI solutions can provide information to avoid unnecessary radiation doses in lung cancer screening [23]. In addition, delivery methods of AI solutions, such as add-on scenarios as concurrent or second reader, stand-alone, triage, and prescreening scenario [24] should be investigated with variable clinical settings, such as preoperative or follow-up examinations for oncology patients or screening for lung cancer or tuberculosis.…”

Section: Discussionmentioning

confidence: 99%

Diagnostic effect of artificial intelligence solution for referable thoracic abnormalities on chest radiography: a multicenter respiratory outpatient diagnostic cohort study

Jin

Kim

et al. 2022

Eur Radiol

View full text Add to dashboard Cite

Objectives We aim ed to evaluate a commercial artificial intelligence (AI) solution on a multicenter cohort of chest radiographs and to compare physicians' ability to detect and localize referable thoracic abnormalities with and without AI assistance. Methods In this retrospective diagnostic cohort study, we investigated 6,006 consecutive patients who underwent both chest radiography and CT. We evaluated a commercially available AI solution intended to facilitate the detection of three chest abnormalities (nodule/masses, consolidation, and pneumothorax) against a reference standard to measure its diagnostic performance. Moreover, twelve physicians, including thoracic radiologists, board-certified radiologists, radiology residents, and pulmonologists, assessed a dataset of 230 randomly sampled chest radiographic images. The images were reviewed twice per physician, with and without AI, with a 4-week washout period. We measured the impact of AI assistance on observer's AUC, sensitivity, specificity, and the area under the alternative free-response ROC (AUAFROC). Results In the entire set (n = 6,006), the AI solution showed average sensitivity, specificity, and AUC of 0.885, 0.723, and 0.867, respectively. In the test dataset (n = 230), the average AUC and AUAFROC across observers significantly increased with AI assistance (from 0.861 to 0.886; p = 0.003 and from 0.797 to 0.822; p = 0.003, respectively). Conclusions The diagnostic performance of the AI solution was found to be acceptable for the images from respiratory outpatient clinics. The diagnostic performance of physicians marginally improved with the use of AI solutions. Further evaluation of AI assistance for chest radiographs using a prospective design is required to prove the efficacy of AI assistance. Key Points • AI assistance for chest radiographs marginally improved physicians’ performance in detecting and localizing referable thoracic abnormalities on chest radiographs. • The detection or localization of referable thoracic abnormalities by pulmonologists and radiology residents improved with the use of AI assistance.

show abstract

Leveraging Deep Object Detection Models for Early Detection of Cancerous Lung Nodules in Chest X-Rays

Mahmud,

Shuvo,

Iqbal

et al. 2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

AI-based improvement in lung cancer detection on chest radiographs: results of a multi-reader study in NLST dataset

Cited by 48 publications

References 25 publications

An Artificial Intelligence–Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study

An Artificial Intelligence–Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study

Diagnostic effect of artificial intelligence solution for referable thoracic abnormalities on chest radiography: a multicenter respiratory outpatient diagnostic cohort study

Leveraging Deep Object Detection Models for Early Detection of Cancerous Lung Nodules in Chest X-Rays

Contact Info

Product

Resources

About