Abstract:IMPORTANCEMost early lung cancers present as pulmonary nodules on imaging, but these can be easily missed on chest radiographs. OBJECTIVE To assess if a novel artificial intelligence (AI) algorithm can help detect pulmonary nodules on radiographs at different levels of detection difficulty.
DESIGN, SETTING, AND PARTICIPANTSThis diagnostic study included 100 posteroanterior chest radiograph images taken between 2000 and 2010 of adult patients from an ambulatory health care center in Germany and a lung image dat… Show more
“…The AUCs reported in our study surpass those reported in previous studies. In a study by Homayounieh et al ., 100 chest radiographs from multiple institutes were evaluated by 9 readers resulting in an increase in mean AUC from 71.7% (unaided read) to 77.2% (aided read) 16 . Jang et al reported an increase in the overall AUC of readers from 67% to 76% when using their DLAD system 17 .…”
Objective: To assess a deep learning-based artificial intelligence model for the detection of pulmonary nodules on chest radiographs and to compare its performance with board-certified human readers.
Methods: For this retrospective study, 308 chest radiographs were obtained between January 2019 to December 2021 from a tertiary care hospital. All radiographs were analyzed using a deep learning AI model called DxNodule AI Screen. Two expert board-certified radiologists established the ground truth, and 11 test readers independently reviewed all radiographs in two sessions (unaided and AI-aided mode) with a washout period of one month.
Results: The standalone model had an AUROC of 0.905 [0.87, 0.94] in detecting pulmonary nodules. The mean AUROC across the 11 readers improved from 0.798 [0.74, 0.86] for unaided interpretation to 0.846 [0.82, 0.880] for AI-aided interpretation. With DxNodule AI Screen, readers were able to identify nodules at the correct locations, which they otherwise missed. The mean specificity, accuracy, PPV, and NPV of the readers improved significantly from 0.87 [0.78, 0.96], 0.78 [0.72, 0.84], 0.77 [0.65, 0.88], and 0.86 [0.81, 0.90] in the unaided session to 0.89 [0.82, 0.96], 0.83 [0.80, 0.85], 0.82 [0.73, 0.9], and 0.89 [0.86, 0.92], respectively in the aided session.
Conclusion: DxNodule AI Screen outperformed human readers in nodule detection performance on chest radiographs, and enhanced human readers' performances when used as an aid.
“…The AUCs reported in our study surpass those reported in previous studies. In a study by Homayounieh et al ., 100 chest radiographs from multiple institutes were evaluated by 9 readers resulting in an increase in mean AUC from 71.7% (unaided read) to 77.2% (aided read) 16 . Jang et al reported an increase in the overall AUC of readers from 67% to 76% when using their DLAD system 17 .…”
Objective: To assess a deep learning-based artificial intelligence model for the detection of pulmonary nodules on chest radiographs and to compare its performance with board-certified human readers.
Methods: For this retrospective study, 308 chest radiographs were obtained between January 2019 to December 2021 from a tertiary care hospital. All radiographs were analyzed using a deep learning AI model called DxNodule AI Screen. Two expert board-certified radiologists established the ground truth, and 11 test readers independently reviewed all radiographs in two sessions (unaided and AI-aided mode) with a washout period of one month.
Results: The standalone model had an AUROC of 0.905 [0.87, 0.94] in detecting pulmonary nodules. The mean AUROC across the 11 readers improved from 0.798 [0.74, 0.86] for unaided interpretation to 0.846 [0.82, 0.880] for AI-aided interpretation. With DxNodule AI Screen, readers were able to identify nodules at the correct locations, which they otherwise missed. The mean specificity, accuracy, PPV, and NPV of the readers improved significantly from 0.87 [0.78, 0.96], 0.78 [0.72, 0.84], 0.77 [0.65, 0.88], and 0.86 [0.81, 0.90] in the unaided session to 0.89 [0.82, 0.96], 0.83 [0.80, 0.85], 0.82 [0.73, 0.9], and 0.89 [0.86, 0.92], respectively in the aided session.
Conclusion: DxNodule AI Screen outperformed human readers in nodule detection performance on chest radiographs, and enhanced human readers' performances when used as an aid.
“…Figure 2 C describes the dCNN architecture used in this study. For full details of the neural network architecture please see Homayounieh et al 2021 Appendix E from which the architecture is sourced [ 22 ]. Fig.…”
Background
Airspace disease as seen on chest X-rays is an important point in triage for patients initially presenting to the emergency department with suspected COVID-19 infection. The purpose of this study is to evaluate a previously trained interpretable deep learning algorithm for the diagnosis and prognosis of COVID-19 pneumonia from chest X-rays obtained in the ED.
Methods
This retrospective study included 2456 (50% RT-PCR positive for COVID-19) adult patients who received both a chest X-ray and SARS-CoV-2 RT-PCR test from January 2020 to March of 2021 in the emergency department at a single U.S. institution. A total of 2000 patients were included as an additional training cohort and 456 patients in the randomized internal holdout testing cohort for a previously trained Siemens AI-Radiology Companion deep learning convolutional neural network algorithm. Three cardiothoracic fellowship-trained radiologists systematically evaluated each chest X-ray and generated an airspace disease area-based severity score which was compared against the same score produced by artificial intelligence. The interobserver agreement, diagnostic accuracy, and predictive capability for inpatient outcomes were assessed. Principal statistical tests used in this study include both univariate and multivariate logistic regression.
Results
Overall ICC was 0.820 (95% CI 0.790–0.840). The diagnostic AUC for SARS-CoV-2 RT-PCR positivity was 0.890 (95% CI 0.861–0.920) for the neural network and 0.936 (95% CI 0.918–0.960) for radiologists. Airspace opacities score by AI alone predicted ICU admission (AUC = 0.870) and mortality (0.829) in all patients. Addition of age and BMI into a multivariate log model improved mortality prediction (AUC = 0.906).
Conclusion
The deep learning algorithm provides an accurate and interpretable assessment of the disease burden in COVID-19 pneumonia on chest radiographs. The reported severity scores correlate with expert assessment and accurately predicts important clinical outcomes. The algorithm contributes additional prognostic information not currently incorporated into patient management.
“…Deep-learning-based algorithms have been developed to detect a single disease such as pneumonia 16 , pneumothorax 17 , tuberculosis 18 , and lung cancer 19 , or multiple diseases at once [20][21][22] . Several commercially available CADs for CXRs have been shown to achieve diagnostic performance comparable to clinicians when used independently (i.e., stand-alone performance) [23][24][25][26][27] and improve clinicians' diagnostic performances (i.e., diagnostic impact) 20,24,28 .…”
Section: Introductionmentioning
confidence: 99%
“…Despite its relevance, localization accuracy is rarely reported in previous evaluations of CADs. Lesion-level diagnostic performance (i.e., whether the model correctly detects a lesion in an image) has been estimated to account for localization but the definition of accurate lesion localization (e.g., any overlap or 20% overlap) is seldom transparently reported 20,22,28,37,38 .…”
While many deep-learning-based computer-aided detection systems (CAD) have been developed and commercialized for abnormality detection in chest radiographs (CXR), their ability to localize a target abnormality is rarely reported. Localization accuracy is important in terms of model interpretability, which is crucial in clinical settings. Moreover, diagnostic performances are likely to vary depending on thresholds which define an accurate localization. In a multi-center, stand-alone clinical trial using temporal and external validation datasets of 1,050 CXRs, we evaluated localization accuracy, localization-adjusted discrimination, and calibration of a commercially available deep-learning-based CAD for detecting consolidation and pneumothorax. The CAD achieved image-level AUROC (95% CI) of 0.960 (0.945, 0.975), sensitivity of 0.933 (0.899, 0.959), specificity of 0.948 (0.930, 0.963), dice of 0.691 (0.664, 0.718), moderate calibration for consolidation, and image-level AUROC of 0.978 (0.965, 0.991), sensitivity of 0.956 (0.923, 0.978), specificity of 0.996 (0.989, 0.999), dice of 0.798 (0.770, 0.826), moderate calibration for pneumothorax. Diagnostic performances varied substantially when localization accuracy was accounted for but remained high at the minimum threshold of clinical relevance. In a separate trial for diagnostic impact using 461 CXRs, the causal effect of the CAD assistance on clinicians’ diagnostic performances was estimated. After adjusting for age, sex, dataset, and abnormality type, the CAD improved clinicians’ diagnostic performances on average (OR [95% CI] = 1.73 [1.30, 2.32]; p < 0.001), although the effects varied substantially by clinical backgrounds. The CAD was found to have high stand-alone diagnostic performances and may beneficially impact clinicians’ diagnostic performances when used in clinical settings.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
