JCS: An Explainable COVID-19 Diagnosis System by Joint Classification and Segmentation

Abstract

Recently, the coronavirus disease 2019 (COVID-19) has caused a pandemic disease in over 200 countries, influencing billions of humans. To control the infection, identifying and separating the infected people is the most crucial step. The main diagnostic tool is the Reverse Transcription Polymerase Chain Reaction (RT-PCR) test. Still, the sensitivity of the RT-PCR test is not high enough to effectively prevent the pandemic. The chest CT scan test provides a valuable complementary tool to the RT-PCR test, and it can identify the patients in the early-stage with high sensitivity. However, the chest CT scan test is usually time-consuming, requiring about 21.5 minutes per case. This paper develops a novel Joint Classification and Segmentation (JCS) system to perform real-time and explainable COVID-19 chest CT diagnosis. To train our JCS system, we construct a large scale COVID-19 Classification and Segmentation (COVID-CS) dataset, with 144,167 chest CT images of 400 COVID-19 patients and 350 uninfected cases. 3,855 chest CT images of 200 patients are annotated with fine-grained pixel-level labels of opacifications, which are increased attenuation of the lung parenchyma. We also have annotated lesion counts, opacification areas, and locations and thus benefit various diagnosis aspects. Extensive experiments demonstrate that the proposed JCS diagnosis system is very efficient for COVID-19 classification and segmentation. It obtains an average sensitivity of 95.0% and a specificity of 93.0% on the classification test set, and 78.5% Dice score on the segmentation test set of our COVID-CS dataset. The COVID-CS dataset and code are available at https://github.com/yuhuan-wu/JCS.

Checklist for responsible deep learning modeling of medical images based on COVID-19 detection studies

“…The content of these articles was again reviewed for changes that had been made. Articles published in journals: [27] , [28] , [29] , [30] , [31] , [32] , [33] , [34] , [35] , [36] , [37] , [38] , preprints: [39] , [40] , [41] , [42] , [43] , [44] , [45] , [46] , [47] , [48] , [49] , [50] , [51] .…”

“…Segmentation was not used in all studies. Lung segmentation is present in works [31] , [32] , [34] , [45] , [50] and lesion segmentation in [32] , [38] , [47] . The lungs are segmented to remove the unnecessary background because, based on medical experience, the lesions caused by COVID-19 are not located outside the lungs.…”

“…Lesion segmentation, also called infection mask, helps to train the model to recognize infected regions and can be beneficial for further model assessment. Only radiologists can prepare trustworthy and high-quality lung lesion masks, not an automatic tool similar to those in the reviewed works [32] , [38] . However, manual mask preparation takes a lot of time and money and requires a high level of consistency among annotators, but, surely, it is the most valuable method for qualitative and quantitative XAI evaluation.…”

“…It is worth noting that some studies have verified the performance of the model considering different number of classes [28] , [33] , [35] , [48] . Binary classification task detecting COVID-19 and non-COVID-19 is the most popular [27] , [28] , [30] , [33] , [35] , [38] , [39] , [41] , [45] , [46] , [48] , [49] . Another frequently used method is three-class classification [28] , [29] , [33] , [35] , [36] , [37] , [47] , [48] , [51] : no infection/no pneumonia, pneumonia (bacterial or non-COVID-19 viral infection), and COVID-19 (COVID-19 viral infection).…”

“…Studies which do not include full information about the number of cases were excluded. Study [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [42] [44] [45] [47] [49] [51] Number of COVID-19 images 250 230 855 345 502 3,389 127 377 112/137 76 358 400 400 58 234 68 829 99 120 269 Total number of images 6523 460 15,959 720 1004 2186 1125 754 366 5949 13,975 750 800 2800 1234 5941 1,865 …”