Evaluation of the Usefulness of CO-RADS for Chest CT in Patients Suspected of Having COVID-19

Abstract

The purpose of this study was to use the Coronavirus Disease 2019 (COVID-19) Reporting and Data System (CO-RADS) to evaluate the chest computed tomography (CT) images of patients suspected of having COVID-19, and to investigate its diagnostic performance and interobserver agreement. The Dutch Radiological Society developed CO-RADS as a diagnostic indicator for assessing suspicion of lung involvement of COVID-19 on a scale of 1 (very low) to 5 (very high). We investigated retrospectively 154 adult patients with clinically suspected COVID-19, between April and June 2020, who underwent chest CT and reverse transcription-polymerase chain reaction (RT-PCR). The patients’ average age was 61.3 years (range, 21–93), 101 were male, and 76 were RT-PCR positive. Using CO-RADS, four radiologists evaluated the chest CT images. Sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were calculated. Interobserver agreement was calculated using the intraclass correlation coefficient (ICC) by comparing the individual reader’s score to the median of the remaining three radiologists. The average sensitivity was 87.8% (range, 80.2–93.4%), specificity was 66.4% (range, 51.3–84.5%), and AUC was 0.859 (range, 0.847–0.881); there was no significant difference between the readers (p > 0.200). In 325 (52.8%) of 616 observations, there was absolute agreement among observers. The average ICC of readers was 0.840 (range, 0.800–0.874; p < 0.001). CO-RADS is a categorical taxonomic evaluation scheme for COVID-19 pneumonia, using chest CT images, that provides outstanding performance and from substantial to almost perfect interobserver agreement for predicting COVID-19.

Diagnostic Performance of CO-RADS and the RSNA Classification System in Evaluating COVID-19 at Chest CT: A Meta-Analysis

“…For the CO-RADS, substantial to almost perfect interobserver agreement has been reported, with ĸ values of 0.648 to 0.773 ( 8 ) and intraclass correlation coefficients of 0.800 to 0.874 ( 20 ). For the RSNA classification system, moderate to substantial interobserver agreement has been reported, with ĸ values of 0.500 ( 23 ) and of 0.570 to 0.663 ( 8 ).…”

“…After reading the full text of the 14 studies, three studies ( 16 - 18 ) were excluded because the diagnostic performance of either CO-RADS or the RSNA classification system was not investigated, one study ( 5 ) was excluded because no data on a per-patient level were reported, and another study ( 19 ) was excluded because there were overlapping data with another study ( 8 ) which comprised a larger number of patients. Nine studies were eventually included ( 8 , 20 - 27 ).…”

“…Figure 2 provides a summary of the QUADAS-2 quality assessments. In one study ( 20 ), it was unclear whether patients were enrolled consecutively or randomly. There was no risk of bias with regard to patient selection in the other studies or with regard to index test.…”

“…Risk of bias with respect to reference test was rated unclear in one study ( 21 ), because it was not clear whether all patients with an initial negative RT-PCR result and a persistent clinical suspicion of COVID-19 underwent repeated RT-PCR testing. In one study ( 20 ), there was potential risk of bias with regard to flow and timing, because the time interval between CT and RT-PCR testing was not reported. There was no risk of bias with regard to flow and timing in the other studies, because the maximum time interval between chest CT and RT-PCR did not exceed seven days ( 22 ).…”

“…Geographical differences, non-reported prevalence of other lung diseases, interobserver variability in chest CT assessment, RT-PCR performance, and some methodological quality issues may have been potential sources of heterogeneity. Note that interobserver agreement varies from substantial to almost perfect for the CO-RADS ( 8 , 20 ) and from moderate to substantial for the RSNA classification system ( 8 , 23 ).…”