Evaluation of an Artificial Intelligence Model for Detection of Pneumothorax and Tension Pneumothorax in Chest Radiographs

Abstract

ImportanceEarly detection of pneumothorax, most often via chest radiography, can help determine need for emergent clinical intervention. The ability to accurately detect and rapidly triage pneumothorax with an artificial intelligence (AI) model could assist with earlier identification and improve care.ObjectiveTo compare the accuracy of an AI model vs consensus thoracic radiologist interpretations in detecting any pneumothorax (incorporating both nontension and tension pneumothorax) and tension pneumothorax.Design, Setting, and ParticipantsThis diagnostic study was a retrospective standalone performance assessment using a data set of 1000 chest radiographs captured between June 1, 2015, and May 31, 2021. The radiographs were obtained from patients aged at least 18 years at 4 hospitals in the Mass General Brigham hospital network in the United States. Included radiographs were selected using 2 strategies from all chest radiography performed at the hospitals, including inpatient and outpatient. The first strategy identified consecutive radiographs with pneumothorax through a manual review of radiology reports, and the second strategy identified consecutive radiographs with tension pneumothorax using natural language processing. For both strategies, negative radiographs were selected by taking the next negative radiograph acquired from the same radiography machine as each positive radiograph. The final data set was an amalgamation of these processes. Each radiograph was interpreted independently by up to 3 radiologists to establish consensus ground-truth interpretations. Each radiograph was then interpreted by the AI model for the presence of pneumothorax and tension pneumothorax. This study was conducted between July and October 2021, with the primary analysis performed between October and November 2021.Main Outcomes and MeasuresThe primary end points were the areas under the receiver operating characteristic curves (AUCs) for the detection of pneumothorax and tension pneumothorax. The secondary end points were the sensitivities and specificities for the detection of pneumothorax and tension pneumothorax.ResultsThe final analysis included radiographs from 985 patients (mean [SD] age, 60.8 [19.0] years; 436 [44.3%] female patients), including 307 patients with nontension pneumothorax, 128 patients with tension pneumothorax, and 550 patients without pneumothorax. The AI model detected any pneumothorax with an AUC of 0.979 (95% CI, 0.970-0.987), sensitivity of 94.3% (95% CI, 92.0%-96.3%), and specificity of 92.0% (95% CI, 89.6%-94.2%) and tension pneumothorax with an AUC of 0.987 (95% CI, 0.980-0.992), sensitivity of 94.5% (95% CI, 90.6%-97.7%), and specificity of 95.3% (95% CI, 93.9%-96.6%).Conclusions and RelevanceThese findings suggest that the assessed AI model accurately detected pneumothorax and tension pneumothorax in this chest radiograph data set. The model’s use in the clinical workflow could lead to earlier identification and improved care for patients with pneumothorax.

Evaluation of an artificial intelligence model for identification of intracranial hemorrhage subtypes on computed tomography of the head

“…As has been noted on an assessment of a similar model for pneumothorax identification, the FDA regulations for a CADt device only permit devices to output the binary classification performance (the AI model assessed here technically outputs a binary classification for each of the four subtypes). 23 A segmentation output could otherwise further assist with explainability by demonstrating the location of the identified intracranial hemorrhage.…”

“…As was noted in the similar assessment of a pneumothorax model, this study therefore establishes the accuracy of the model but does not assess its impact on the clinical workflow including for case prioritization and patient outcomes. 23 This initial step is necessary to ensure the model has the potential to provide clinical benefit. Further evaluation will be required moving forward to prove such benefit.…”

Incidental radiological findings during clinical tuberculosis screening in Lesotho and South Africa: a case series

“…Similarly, Annalise CXR V.1.2 (Annalise-AI, Sydney, Australia) has shown promising results, being validated for 124 clinical CXR findings in a multireader, multicase study [ 28 ]. It is also the first US Food and Drug Administration (FDA)-cleared model that successfully distinguishes a tension pneumothorax from any pneumothorax [ 29 ].…”

Segmentation of Pneumothorax on Chest CTs Using Deep Learning Based on Unet-Resnet-50 Convolutional Neural Network Structure

“…Fully convolutional networks, along with their subsequent extensions such as U-Net, have demonstrated remarkable efficacy in the segmentation of medical images, even when trained with a limited quantity of labeled data [3,4]. U-Net and U-Net-like models have shown effective in segmenting many anatomical structures such as the lungs, pulmonary nodules, clavicles, brain, heart, and prostate [1][2][3]5] Various U-Net-inspired convolutional neural network (CNN) techniques have been suggested for pneumothorax identification using image-level annotation (classification) on chest X-rays [1,3,[6][7][8][9][10][11][12][13][14][15][16]. Lesion semantic segmentation in medical imaging is a crucial tool for facilitating lesion analysis and therapy planning.…”