Reliability of on-call radiology residents’ interpretation of 64-slice CT pulmonary angiography for the detection of pulmonary embolism

Joshi, Rohit; Wu, Ke-Li; Kaicker, Jatin; Choudur, Hema

doi:10.1177/0284185113506135

Cited by 16 publications

(21 citation statements)

References 11 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Studies has shown that there can be up to a 13% discrepancy rate between overnight radiologists and daytime faculty. [5][6][7] Increasing pressure is placed on hospital systems to provide 24-7 access to advanced imaging and to ensure that the results of urgent findings, such as PE, are rapidly and accurately communicated to the referring clinician. 8,9 However, providing rapid and accurate diagnostic imaging is increasingly difficult to sustain for many medical systems and radiology providers as utilization has expanded; for example, CTPA usage alone in the emergency setting has increased 27-fold over the past 2 decades.…”

Section: Introductionmentioning

confidence: 99%

PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging

et al. 2020

View full text Add to dashboard Cite

Pulmonary embolism (PE) is a life-threatening clinical problem and computed tomography pulmonary angiography (CTPA) is the gold standard for diagnosis. Prompt diagnosis and immediate treatment are critical to avoid high morbidity and mortality rates, yet PE remains among the diagnoses most frequently missed or delayed. In this study, we developed a deep learning model-PENet, to automatically detect PE on volumetric CTPA scans as an end-to-end solution for this purpose. The PENet is a 77-layer 3D convolutional neural network (CNN) pretrained on the Kinetics-600 dataset and fine-tuned on a retrospective CTPA dataset collected from a single academic institution. The PENet model performance was evaluated in detecting PE on data from two different institutions: one as a hold-out dataset from the same institution as the training data and a second collected from an external institution to evaluate model generalizability to an unrelated population dataset. PENet achieved an AUROC of 0.84 [0.82-0.87] on detecting PE on the hold out internal test set and 0.85 [0.81-0.88] on external dataset. PENet also outperformed current state-of-the-art 3D CNN models. The results represent successful application of an end-to-end 3D CNN model for the complex task of PE diagnosis without requiring computationally intensive and time consuming preprocessing and demonstrates sustained performance on data from an external institution. Our model could be applied as a triage tool to automatically identify clinically important PEs allowing for prioritization for diagnostic radiology interpretation and improved care pathways via more efficient diagnosis.npj Digital Medicine (2020) 3:61 ; https://doi.

show abstract

Section: Introductionmentioning

confidence: 99%

PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging

et al. 2020

View full text Add to dashboard Cite

show abstract

“…There have been several previous studies addressing the degree of agreement between experienced radiologists and radiology residents’ interpretations of CTPA in an emergency context [ 23 – 25 ]. Our study shows a good IOA between the on-call radiology residents’ and the staff radiologists’ interpretation for PE with an overall agreement of 91.4% (kappa of 0.81).…”

Section: Discussionmentioning

confidence: 99%

Interobserver Agreement between On-Call Radiology Resident and General Radiologist Interpretations of CT Pulmonary Angiograms and CT Venograms

et al. 2015

View full text Add to dashboard Cite

ObjectivesTo evaluate the interobserver agreement (IOA) between the initial radiology resident and the final staff radiologist reports of combined computed tomographic pulmonary angiograms (CTPA) and computed tomographic venograms (CTV) performed during on-call hours.Materials and MethodsApproval by the institutional review board was obtained. Six-hundred and ninety-six consecutive studies (CTPA or CTPA with CTV) performed during on-call hours and interpreted by 30 residents were identified. Radiology residents’ reports were compared to the final staff reports. Three tests outcomes were considered (positive, P; negative, N; indeterminate, I). Discordant cases were reviews by a chest radiologist.ResultsCTPAs were reported by staff radiologists as positive for pulmonary embolism (PE) in 18% (126/694), with a kappa of 0.81 (95% CI 0.77-0.86) with 3 outcomes (P, N, I), and a kappa of 0.89 (95% CI 0.85-0.94) with 2 outcomes (P, N). Regarding PE location, good concordance was observed for positive studies, with a kappa of 0.86 (95% CI 0.78 – 0.95). CTVs were reported as positive by staff radiologists in 8.5% (33/388), with a kappa of 0.66 (95% CI 0.55-0.77) with 3 outcomes (P, N, I), and a kappa of 0.89 (95% CI 0.8-1.0) with 2 outcomes (P, N). The IOA between residents and staff radiologists increased with increasing residency year level for CTPAs, but did not for CTVs.ConclusionsVery good and good IOA were observed between resident and staff radiologist interpretations for CTPA and CTV, respectively, with tendency towards improved IOA as residency level of training increased for CTPA, but not for CTV.

show abstract

“…In addition, different outcome measures are used to assess improvements in diagnostic ability over time, including sensitivity/specificity, ROC-analysis and interreader agreement, making direct comparison among studies more difficult. There appears to be a general agreement that the amount of training is an important factor to achieve a high diagnostic accuracy (77,78,81), while the effects of previous radiologic experience on the ability to learn new radiologic methods have not been established (14,80). A few studies have shown a positive effect on the review time, but not on diagnostic accuracy (78,80).…”

Section: Pedagogical Aspectsmentioning

confidence: 99%

“…In studies on MRI in the detection of APE, primarily senior, experienced radiologists have reviewed the exams (65,(82)(83)(84)(85). However, in a clinical setting residents are commonly the primary reviewers (81). When introducing a new method, knowledge of the learning curves is of particular interest.…”

Section: Pedagogical Aspectsmentioning

confidence: 99%

Do radiologists detect chronic thromboembolic disease on computed tomography?

et al. 2019

View full text Add to dashboard Cite

Background: Acute pulmonary embolism (APE) is a potentially severe medical condition with blood clots obstructing the pulmonary arterial vasculature. In most cases the APE resolves without any sequelae after anticoagulation therapy. In some patients, however, the emboli do not resolve upon treatment and the remnants cause increased vascular resistance, a condition known as chronic thromboembolic pulmonary hypertension (CTEPH). Both APE and CTEPH have a non-specific clinical presentation and imaging is an important part of the diagnosis. In APE computed tomography pulmonary angiography (CTPA) is the diagnostic gold standard, although the method is not suitable for all patients. CTPA has a high specificity for CTEPH, but the sensitivity remains under debate. At present CTPA is not recommended as a first line test among patients with a clinical suspicion of CTEPH. Purpose: To investigate unestablished imaging modalities in the diagnosis of APE (Study I) and CTEPH (Study III) including learning aspects (Study II) and knowledge (Study IV) of theses among radiologists. Regarding APE we studied magnetic resonance imaging (MRI) and in CTEPH we studied CTPA. Material and methods: Studies I-II were based on a prospective collection of 70 unenhanced MRI exams with CTPA as the gold standard. In Studies III-IV we used a retrospective material based on 43 CTPA exams from patients with confirmed CTEPH referred for presurgical assessment at a specialist centre, with a matched control with suspected APE. Results: All MRI exams were of diagnostic quality. Specificity was 100% for both readers and sensitivity 90% and 93% respectively with a nearly perfect inter-reader agreement (kappa 0.97) (Study I). Residents interpreting the MRI exams within the training program reached a clinically acceptable level after approximately 50 examinations and review time was halved during the training program (Study II). The sensitivity for CTEPH on CTPA reviewed by two experts was 100% and the specificity 100% (Study III), while the sensitivity based on the original reports from the same cases was 26% (Study IV). Conclusions: Unenhanced MRI has a high sensitivity and specificity for APE (Study I) and residents can learn to interpret such exams by using a self-directed training program (Study II). Enhanced CTPA has a high sensitivity when reviewed by experienced radiologists (Study III), but among radiologists in general the sensitivity is low (Study IV). LIST OF SCIENTIFIC PAPERS I. Detection of pulmonary embolism using repeated MRI acquisitions without respiratory gating: A preliminary study.

show abstract

Reliability of on-call radiology residents’ interpretation of 64-slice CT pulmonary angiography for the detection of pulmonary embolism

Abstract: The rate of discrepancy fell steeply between the second and fifth year of the residents training from 18.5% to 6.9%. Our study suggests that it is reasonable to have on-call radiology residents perform the preliminary interpretations of 64-slice CT for PE studies.

Cited by 16 publications

References 11 publications

PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging

PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging

Interobserver Agreement between On-Call Radiology Resident and General Radiologist Interpretations of CT Pulmonary Angiograms and CT Venograms

Do radiologists detect chronic thromboembolic disease on computed tomography?

Contact Info

Product

Resources

About