Classification of Video Capsule Endoscopy Images Using Visual Transformers

Lima, Daniel Lopes Soares; Pessoa, Alexandre; Paiva, Anselmo Cardoso de; Cunha, António; Bráz, Geraldo; Almeida, João Dallyson Sousa de

doi:10.1109/bhi56158.2022.9926791

Cited by 4 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The transformer is currently a state-of-the-art model for computer vision and NLP tasks. As a result, recent works [13,15] have introduced the application of transformers for processing and analyzing WCE images. These studies have demonstrated the effectiveness of utilizing the transformer architecture in achieving high performance when applied to WCE images.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Video Analysis of Small Bowel Capsule Endoscopy Using a Transformer Network

Oh,

Kim

et al. 2023

Diagnostics

View full text Add to dashboard Cite

Although wireless capsule endoscopy (WCE) detects small bowel diseases effectively, it has some limitations. For example, the reading process can be time consuming due to the numerous images generated per case and the lesion detection accuracy may rely on the operators’ skills and experiences. Hence, many researchers have recently developed deep-learning-based methods to address these limitations. However, they tend to select only a portion of the images from a given WCE video and analyze each image individually. In this study, we note that more information can be extracted from the unused frames and the temporal relations of sequential frames. Specifically, to increase the accuracy of lesion detection without depending on experts’ frame selection skills, we suggest using whole video frames as the input to the deep learning system. Thus, we propose a new Transformer-architecture-based neural encoder that takes the entire video as the input, exploiting the power of the Transformer architecture to extract long-term global correlation within and between the input frames. Subsequently, we can capture the temporal context of the input frames and the attentional features within a frame. Tests on benchmark datasets of four WCE videos showed 95.1% sensitivity and 83.4% specificity. These results may significantly advance automated lesion detection techniques for WCE images.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Furthermore, the vision Transformer (ViT), a model that modified the original transformer for computer vision, has also performed well in image classification [12]. Because ViT performs well in computer vision tasks, some studies have employed the transformer architecture to analyze WCE images [13][14][15].…”

Section: Introductionmentioning

confidence: 99%