Benchmarking Deep Learning Models and Automated Model Design for COVID-19 Detection with Chest CT Scans

Abstract

COVID-19 pandemic has spread all over the world for months. As its transmissibility and high pathogenicity seriously threaten people's lives, the accurate and fast detection of the COVID-19 infection is crucial. Although many recent studies have shown that deep learning based solutions can help detect COVID-19 based on chest CT scans, there lacks a consistent and systematic comparison and evaluation on these techniques. In this paper, we first build a clean and segmented CT dataset called Clean-CC-CCII by fixing the errors and removing some noises in a large CT scan dataset CC-CCII with three classes: novel coronavirus pneumonia (NCP), common pneumonia (CP), and normal controls (Normal). After cleaning, our dataset consists of a total of 340,190 slices of 3,993 scans from 2,698 patients. Then we benchmark and compare the performance of a series of state-of-the-art (SOTA) 3D and 2D convolutional neural networks (CNNs). The results show that 3D CNNs outperform 2D CNNs in general. With extensive effort of hyperparameter tuning, we find that the 3D CNN model DenseNet3D121 achieves the highest accuracy of 88.63% (F1-score is 88.14% and AUC is 0.940), and another 3D CNN model ResNet3D34 achieves the best AUC of 0.959 (accuracy is 87.83% and F1-score is 86.04%). We further demonstrate that the mixup data augmentation technique can largely improve the model performance. At last, we design an automated deep learning methodology to generate a lightweight deep learning model MNas3DNet41 that achieves an accuracy of 87.14%, F1-score of 87.25%, and AUC of 0.957, which are on par with the best models made by AI experts. The automated deep learning design is a promising methodology that can help health-care professionals develop effective deep learning models using their private data sets. Our Clean-CC-CCII dataset and source code are available at: https://github.com/arthursdays/HKBU\_HPML\_COVID-19.

Applications of artificial intelligence in battling against covid-19: A literature review

“…In [254] , three CNN algorithms (ResNet50, InceptionV3 and Inception-ResNetV2) are proposed to process X-Ray radiographs. In [255] , a CT dataset is introduced and a series of convolutional neural networks are used on the data. In [256] , a CNN combined with KNN is used to classify CT images.…”

“…A set of 48262 CT scan images from 282 normal and 15589 patients are collected and shared in [631] . In [255] , a clean and segmented CT dataset called Clean-CC-CCII is presented by fixing the errors and removing some noises in a large CT scan dataset CC-CCII with three classes: novel coronavirus pneumonia (NCP), common pneumonia (CP), and normal controls (Normal). After cleaning, the dataset consists of a total of 340,190 slices of 3,993 scans from 2,698 patients.…”

Chest Computed Tomography Scan Images for Classification of Coronavirus by Enhanced Convolutional Neural Network and Gabor Filter

“…It trains the VGG16 network which is 9 times faster than and 213 faster at test time. Xien He at el [22] proposed an automated model design for covid19 detection in chest CT scans. In this work he proved that 3D CNN outperforms the 2D CNN.…”

AI-Empowered Computational Examination of Chest Imaging for COVID-19 Treatment: A Review

“…Transfer learning is still the most common technique among the diverse methods to detect COVID-19 from lung CT images (Anwar and Zakir, 2020;He et al, 2020b;Chowhury et al, 2020;Soares et al, 2020;Wang S. et al, 2021). Particularly, previous studies (He et al, 2020a;Ardakani et al, 2020) have built a benchmark to evaluate state-of-the-art 2D and 3D CNN models (e.g., DenseNet and ResNet) for lung CT slides classification. It is worth mentioning that in the study of Wang S. et al (2021), the model also performed redetection on the results of the nucleic acid testing.…”