Objective
To conduct a multireader validation study to evaluate the interobserver variability and the diagnostic accuracy for the lung involvement by COVID-19 of COVID-19 Reporting and Data System (CO-RADS) score.
Methods
This retrospective study included consecutive symptomatic patients who underwent chest CT and reverse transcriptase-polymerase chain reaction (RT-PCR) from March 2020 to May 2020 for suspected COVID-19. Twelve readers with different levels of expertise independently scored each CT using the CO-RADS scheme for detecting pulmonary involvement by COVID-19. Receiver operating characteristic (ROC) curves were computed to investigate diagnostic yield. Fleiss’ kappa statistics was used to evaluate interreader agreement.
Results
A total of 572 patients (mean age, 63 ± 20 [standard deviation]; 329 men; 142 patients with COVID-19 and 430 patients without COVID-19) were evaluated. There was a moderate agreement for CO-RADS rating among all readers (Fleiss’
K
= 0.43 [95% CI 0.42–0.44]) with a substantial agreement for CO-RADS 1 category (Fleiss’
K
= 0.61 [95% CI 0.60–0.62]) and moderate agreement for CO-RADS 5 category (Fleiss’
K
= 0.60 [95% CI 0.58–0.61]). ROC analysis showed the CO-RADS score ≥ 4 as the optimal threshold, with a cumulative area under the curve of 0.72 (95% CI 66–78%), sensitivity 61% (95% CI 52–69%), and specificity 81% (95% CI 77–84%).
Conclusion
CO-RADS showed high diagnostic accuracy and moderate interrater agreement across readers with different levels of expertise. Specificity is higher than previously thought and that could lead to reconsider the role of CT in this clinical setting.
Key Points
• COVID-19 Reporting and Data System (CO-RADS) demonstrated a good diagnostic accuracy for lung involvement by COVID-19 with an average AUC of 0.72 (95% CI 67
–
75%).
•
When a threshold of ≥ 4 was used, sensitivity and specificity were 61% (95% CI 52
–
69%) and 81% (95% CI 76
–
84%), respectively.
•
There was an overall moderate agreement for CO-RADS rating across readers with different levels of expertise (Fleiss’ K = 0.43 [95% CI 0.42
–
0.44]).
Electronic supplementary material
The online version of this article (10.1007/s00330-020-07273-y) contains supplementary material, which is available to authorized users.
The typical findings on CT in patients affected by novel COVID-19 (coronavirus disease 2019) pneumonia are characterized by ground-glass opacities and/or air space consolidation mainly bilateral and peripherical in distribution, including crazy paving pattern and reversed halo sign. We hereby describe a case of an adolescent male tested positive for COVID-19 with mild respiratory symptoms and presenting with pneumomediastinum as the only CT finding.
Introduction: A more structured role of radiographers is advisable to speed up the management of patients with suspected COVID-19. The purpose of our study was to evaluate the diagnostic performance of radiographers in the detection of COVID-19 pneumonia on chest CT using CO-RADS descriptors. Methods: CT images of patients who underwent RT-PCR and chest CT due to COVID-19 suspicion between March and July 2020 were analysed retrospectively. Six readers, including two radiologists, two highly experienced radiographers and two less experienced radiographers, independently scored each CT using the CO-RADS lexicon. ROC curves were used to investigate diagnostic accuracy, and Fleiss'k statistics to evaluate inter-rater agreement. Results: 714 patients (419 men; 295 women; mean age: 64 years ±19SD) were evaluated. CO-RADS> 3 was identified as optimal diagnostic threshold. Highly experienced radiographers achieved an average sensitivity of 58.7% (95%CI: 52.5e64.7), an average specificity of 81.8% (95%CI: 77.9e85.2), and a mean AUC of 0.72 (95%CI: 0.68e0.75). Among less experienced radiographers, an average sensitivity of 56.3% (95%CI: 50.1e62.2) and an average specificity of 81.5% (95%CI: 77.6e84.9) were observed, with a mean AUC of 0.71 (95%CI: 0.68e0.74). Consultant radiologists achieved an average sensitivity of 60.0% (95%CI: 53.7e65.8), an average specificity of 81.7% (95%CI: 77.8e85.1), and a mean AUC of 0.73 (95%CI: 0.70 e0.77). Conclusion: Radiographers can adequately recognise the classic appearances of COVID-19 on CT, as described by the CO-RADS assessment scheme, in a way comparable to expert radiologists. Implications for practice: Radiographers, as the first healthcare professionals to evaluate CT images in patients with suspected SARS-CoV-2 infection, could diagnose COVID-19 pneumonia by means of a categorical reporting scheme at CT in a reliable way, hence playing a primary role in the early management of these patients.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.