RSNA and BSTI grading systems of COVID-19 pneumonia: comparison of the diagnostic performance and interobserver agreement

Abstract

Background This study aimed to compare the performance and interobservers agreement of cases with findings on chest CT based on the British Society of Thoracic Imaging (BSTI) guideline statement of COVID-19 and the Radiological Society of North America (RSNA) expert consensus statement. Methods In this study, 903 patients who had admitted to the emergency department with a pre-diagnosis of COVID-19 between 1 and 18 July 2020 and had chest CT. Two radiologists classified the chest CT findings according to the RSNA and BSTI consensus statements. The performance, sensitivity and specificity values of the two classification systems were calculated and the agreement between the observers was compared by using kappa analysis. Results Considering RT-PCR test result as a gold standard, the sensitivity, specificity and positive predictive values were significantly higher for the two observers according to the BSTI guidance statement and the RSNA expert consensus statement (83.3%, 89.7%, 89.0%; % 81.2,% 89.7,% 88.7, respectively). There was a good agreement in the PCR positive group (κ: 0.707; p < 0.001 for BSTI and κ: 0.716; p < 0.001 for RSNA), a good agreement in the PCR negative group (κ: 0.645; p < 0.001 for BSTI and κ: 0.743; p < 0.001 for RSNA) according to the BSTI and RSNA classification between the two radiologists. Conclusion As a result, RSNA and BSTI statement provided reasonable performance and interobservers agreement in reporting CT findings of COVID-19. However, the number of patients defined as false negative and indeterminate in both classification systems is at a level that cannot be neglected.

RETRACTED: The Relation of Polymerase Chain Reaction and Computed Tomography Infiltrations of Suspected Covid-19 Patients in Emergency Department

“…T-CT, as a diagnostic tool, is subject to the expertise and reporting accuracy of the observer. The diagnostic performance of T-CT with RSNA categorization using the typical category as a threshold has been reported in various studies, showing a wide range of sensitivity (64%-92%), speci city (73-97%), PPV (23-97%), NPV (79-95%), and accuracy (78%-91%) [14][15][16][17][18]. Our study's performance measurements fall within these ranges.…”

Diagnostic performance of standardized typical CT findings for COVID-19: a systematic review and meta-analysis

“…There were 18 studies from developing countries (4 studies from lower-middle-income and 14 studies from upper-middle-income countries) [ 18 , 19 , 22 , 23 , 30 – 32 , 35 , 37 , 38 , 43 – 45 , 47 , 48 , 50 , 51 , 58 ] and 24 studies from developed countries (high income) [ 9 , 20 , 21 , 24 – 29 , 33 , 34 , 36 , 39 – 42 , 46 , 49 , 52 – 57 ]. The included studies were predominantly from Europe (24 studies) [ 9 , 20 , 21 , 23 – 27 , 29 , 31 , 32 , 34 , 35 , 37 , 38 , 41 , 42 , 47 – 49 , 52 , 55 – 57 ], followed by the Americas (10 studies), Asia (seven studies), and Africa (one study). Nineteen studies for assessing the interobserver agreement are summarized in Additional file 1 : Table 1 .…”

“…Interobserver agreement for the typical CT findings was reported in 11 studies: seven studies reported diagnostic accuracy and the interobserver agreement together [ 9 , 20 , 22 , 28 , 37 , 45 , 51 ] and four studies solely reported the interobserver agreement [ 59 , 60 , 63 , 64 ]. The reported κ values ranged from 0.52 to 0.93.…”

Do COVID-19 CT features vary between patients from within and outside mainland China? Findings from a meta-analysis

“…Radiologists could accurately identify patients with COVID-19 positive using standard CT diagnostic procedures; however, there were obvious subjective factors and a lack of objective quantitative standards in the quantitative score of CT images to identify COVID-19 at the moment. Hence, the Radiological Society of North America (RSNA), the British Society of Thoracic Imaging (BSTI), the Dutch Radiological Society (COVID-19 Reporting and Data System, CO-RADS), etc., issued their expert consensus document to standardize COVID-19 CT findings and reports (107)(108)(109), among which RSNA is the most accepted structural reporting system (108,110,111). There is a bias in sensitivity values in most chest CT studies that do not include RSNA structural reporting system or others.…”

“…There is a bias in sensitivity values in most chest CT studies that do not include RSNA structural reporting system or others. Uysal et al (112) reported that one-quarter of patients with asymptomatic COVID-19 had a normal chest CT. Kavak et al (109) emphasized that false-negative patients should not be neglected in the RSNA and BSTI systems. Although the near perfect consistent and reproducible in the positive predictive value were observed in the system of RSNA and CO-RADS ′ O ′ Neill et al (108) found that end-users preferred RSNA system for it's reporting language.…”