xViTCOS: Explainable Vision Transformer Based COVID-19 Screening Using Radiography

Mondal, Arnab Kumar; Bhattacharjee, Arnab; Singla, Parag; Ap, Prathosh

doi:10.36227/techrxiv.14912367.v1

Cited by 9 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [43] propose a convolutional CapsNet for COVID-19 detection from chest X-ray images in binary as well as multi-class classification settings. xViTCOS [44] propose a vision transformer based deep neural classifier for COVID-19 prognosis.…”

Section: Related Workmentioning

confidence: 99%

COVID-19 prognosis using limited chest X-ray images

Mondal

2022

Applied Soft Computing

Self Cite

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

COVID-19 prognosis using limited chest X-ray images

Mondal

2022

Applied Soft Computing

Self Cite

View full text Add to dashboard Cite

“…They need a vast receptive eld to capture long-range dependencies, which means developing large kernels or highly massive networks, resulting in an extremely complex model that is challenging to train. [9]. To overcome these drawbacks of CNN, some researchers have used other architectures, such as Capsule Neural Networks (Capsnets) [24] and ViT [8], for COVID-19 classi cation, which differs from the traditional CNN networks.…”

Section: Related Workmentioning

confidence: 99%

“…The trained model performed better than CNN, such as E cientNet-B0, Inception-V3, and ResNet-50 in a multi-classi cation challenge, with 92% accuracy and 98% AUC. Mondal et al [9] (2) iCTCF [30], which comes from a total of 1521 patients in two hospitals of Huazhong University of Science and Technology, China, including 894 COVID-19 pneumonia cases (including mild, severe, and critical cases), 328 novel coronavirus-negative patients (control group), and 299 patients with suspected COVID-19; (3) COVID-CTSet [31], which comes from the dataset of Negin Medical Center in Sari, Iran, including 377 patients with con rmed COVID-19, 95 novel coronavirus-negative patients, and 282 other pneumonia patients; and the remaining were collected from (4) TCIA [32], (5) COVID-19 Infection Segmentation Dataset [33], (6) LIDC-IDRI [34], (7) Radiopaedia [35], (8) and MosMedData [36]. In Table 1, MUST-COVID-19 contains images of about three classes, with 80 percent of images employed for training and veri cation and 20 percent for model testing.…”

Section: Related Workmentioning

confidence: 99%

“…By February 24, 2022, it had infected more than 430,220,905 individuals and caused the deaths of 5,936,914 patients worldwide [2]. Because of its rapid mutation and the many powerful immune escape variants that have classi cation results than CNN in the eld of CT image classi cation [9]. Motivated by this, we conduct this study with the following signi cant contributions:…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

STCovidNet: Automatic Detection Model of Novel Coronavirus Pneumonia Based on Swin Transformer

Wang

Zhang

Tian

2022

Preprint

View full text Add to dashboard Cite

The novel coronavirus disease 2019 (COVID-19) has emerged as an enormous challenge facing China today. Preventive Medicine physicians and Artificial Intelligence (AI) researchers try to improve the ability to early automatic warning of coronavirus infections, promote epidemic prevention, and reduce medical costs using deep learning methods. In this work, we build an extensive database of chest computed tomography (CT) scans with image data from domestic and international open-source medical datasets. Swin Transformer is chosen as the backbone network to establish a model (STCovidNet) for the prediction of COVID-19. We then compare the performance of our technique against that of Vision Transformer (ViT) and Convolutional Neural Network (CNN). Next, to visualize our model's high-dimensional outputs in 2-dimensional space, we apply t-distributed stochastic neighbor embedding (t-SNE) as the dimension-reduction strategy. Finally, we employ gradient-weighted class activation mapping (Grad-CAM) to present a class activation map. The results indicate that STCovidNet’s performance surpasses ViT and CNN with a 0.9811 AUC and 0.9858 accuracy score. Our network outperforms previous techniques to reduce intra-class variability and generate well-separated feature embedding. The CAM figure illustrates that the decision region corresponds to radiologists' detecting spots. The suggested method can be an effective way of catching COVID-19 instances.

show abstract

“…Krishnan et al [17] and Park et al [39] utilize ViT-based models to achieve higher COVID-19 classification accuracy through CXR images. COVID-Transformer [48] and xViTCOS [33] have been proposed to further improve classification accuracy and focus on diagnosis-related regions. However, there is still much room for improvement to train ViT models in a small dataset, such as medical imaging dataset.…”

Section: Vision Transformermentioning

confidence: 99%

Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learning

Ma¹,

Zhang²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Learning harmful shortcuts such as spurious correlations and biases prevents deep neural networks from learning the meaningful and useful representations, thus jeopardizing the generalizability and interpretability of the learned representation. The situation becomes even more serious in medical imaging, where the clinical data (e.g., MR images with pathology) are limited and scarce while the reliability, generalizability and transparency of the learned model are highly required. To address this problem, we propose to infuse human experts' intelligence and domain knowledge into the training of deep neural networks. The core idea is that we infuse the visual attention information from expert radiologists to proactively guide the deep model to focus on regions with potential pathology and avoid being trapped in learning harmful shortcuts. To do so, we propose a novel eye-gaze-guided vision transformer (EG-ViT) for diagnosis with limited medical image data. We mask the input image patches that are out of the radiologists' interest and add an additional residual connection in the last encoder layer of EG-ViT to maintain the correlations of all patches. The experiments on two public datasets of INbreast and SIIM-ACR demonstrate our EG-ViT model can effectively learn/transfer experts' domain knowledge and achieve much better performance than baselines. Meanwhile, it successfully rectifies the harmful shortcut learning and significantly improves the EG-ViT model's interpretability. In general, EG-ViT takes the advantages of both human expert's prior knowledge and the power of deep neural networks. This work opens new avenues for advancing current artificial intelligence paradigms by infusing human intelligence.

show abstract

xViTCOS: Explainable Vision Transformer Based COVID-19 Screening Using Radiography

Cited by 9 publications

References 0 publications

COVID-19 prognosis using limited chest X-ray images

COVID-19 prognosis using limited chest X-ray images

STCovidNet: Automatic Detection Model of Novel Coronavirus Pneumonia Based on Swin Transformer

Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learning

Contact Info

Product

Resources

About