Segtransvae: Hybrid Cnn - Transformer with Regularization for Medical Image Segmentation

Pham, Quan-Dung; Nguyen-Truong, Hai; Phuong, Nam Nguyen; Nguyen, Khoa

doi:10.1109/isbi52829.2022.9761417

Cited by 18 publications

(17 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with the UNETR [ 38 ], the recent variant, that is, SWIN UNETR [ 34 ], has 61.98 million parameters. The Segtransvae [ 31 ] has 44.7 million parameters. The BTSWIN-UNet model [ 28 ] has 35.6 million parameters that are higher than other SWIN transformer–based models but much smaller than the UNETR.…”

Section: Resultsmentioning

confidence: 99%

Artificial Intelligence–Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review

Ali,

Qureshi,

Shah

2023

JMIR Med Inform

View full text Add to dashboard Cite

Background Transformer-based models are gaining popularity in medical imaging and cancer imaging applications. Many recent studies have demonstrated the use of transformer-based models for brain cancer imaging applications such as diagnosis and tumor segmentation. Objective This study aims to review how different vision transformers (ViTs) contributed to advancing brain cancer diagnosis and tumor segmentation using brain image data. This study examines the different architectures developed for enhancing the task of brain tumor segmentation. Furthermore, it explores how the ViT-based models augmented the performance of convolutional neural networks for brain cancer imaging. Methods This review performed the study search and study selection following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. The search comprised 4 popular scientific databases: PubMed, Scopus, IEEE Xplore, and Google Scholar. The search terms were formulated to cover the interventions (ie, ViTs) and the target application (ie, brain cancer imaging). The title and abstract for study selection were performed by 2 reviewers independently and validated by a third reviewer. Data extraction was performed by 2 reviewers and validated by a third reviewer. Finally, the data were synthesized using a narrative approach. Results Of the 736 retrieved studies, 22 (3%) were included in this review. These studies were published in 2021 and 2022. The most commonly addressed task in these studies was tumor segmentation using ViTs. No study reported early detection of brain cancer. Among the different ViT architectures, Shifted Window transformer–based architectures have recently become the most popular choice of the research community. Among the included architectures, UNet transformer and TransUNet had the highest number of parameters and thus needed a cluster of as many as 8 graphics processing units for model training. The brain tumor segmentation challenge data set was the most popular data set used in the included studies. ViT was used in different combinations with convolutional neural networks to capture both the global and local context of the input brain imaging data. Conclusions It can be argued that the computational complexity of transformer architectures is a bottleneck in advancing the field and enabling clinical transformations. This review provides the current state of knowledge on the topic, and the findings of this review will be helpful for researchers in the field of medical artificial intelligence and its applications in brain cancer.

show abstract

Section: Resultsmentioning

confidence: 99%

Artificial Intelligence–Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review

Ali,

Qureshi,

Shah

2023

JMIR Med Inform

View full text Add to dashboard Cite

show abstract

“…We compare our proposed CKD-TransBTS model with several SOTA models, including six CNN-based models (VNet [45], ResUNet [46], LSTM-CNN [47], UNet++ [48] AttentionUNet [49] and DynUNet [20]) and six transformerbased models (TransBTS [28], TransUNet [27], UNETR [7], VTNet [30], SegTransVAE [50] and Swin UNETR [32]). For each baseline model in this experiment, we directly run the code if it has been released.…”

Section: B Comparisons With Sota Modelsmentioning

confidence: 99%

CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer With Modality-Correlated Cross-Attention for Brain Tumor Segmentation

Lin

et al. 2023

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Brain tumor segmentation (BTS) in magnetic resonance image (MRI) is crucial for brain tumor diagnosis, cancer management and research purposes. With the great success of the ten-year BraTS challenges as well as the advances of CNN and Transformer algorithms, a lot of outstanding BTS models have been proposed to tackle the difficulties of BTS in different technical aspects. However, existing studies hardly consider how to fuse the multimodality images in a reasonable manner. In this paper, we leverage the clinical knowledge of how radiologists diagnose brain tumors from multiple MRI modalities and propose a clinical knowledge-driven brain tumor segmentation model, called CKD-TransBTS. Instead of directly concatenating all the modalities, we re-organize the input modalities by separating them into two groups according to the imaging principle of MRI. A dual-branch hybrid encoder with the proposed modality-correlated crossattention block (MCCA) is designed to extract the multimodality image features. The proposed model inherits the strengths from both Transformer and CNN with the local feature representation ability for precise lesion boundaries and long-range feature extraction for 3D volumetric images. To bridge the gap between Transformer and CNN features, we propose a Trans&CNN Feature Calibration block (TCFC)

show abstract

“…Transformers were originally proposed for Natural Language Processing 19 , 20 and Text Embedding 21 . As researchers continue to explore, Transformers can be applied not only to object detection 22 and image classification 23 – 25 , but also to semantic segmentation 26 and medical image segmentation 27 , 28 . Based on the powerful global modeling ability of Transformer, we introduce Transformer into combined query image retrieval.…”

Section: Related Workmentioning

confidence: 99%

Composed query image retrieval based on triangle area triple loss function and combining CNN with transformer

Zhang

Wang

Cheng

2022

Sci Rep

View full text Add to dashboard Cite

The existing typical combined query image retrieval methods adopt Euclidean distance as sample distance measurement method, and the model trained by triple loss function blindly pursues absolute distance between samples, resulting in unsatisfactory image retrieval performance. Meanwhile, these methods singularly adopt Convolutional Neural Network (CNN) to extract reference image features. However, receptive field of convolution operation has the characteristics of locality, which is easy to cause the loss of edge feature information of reference images. In view of shortcomings of these methods, the following improvements are proposed in this paper: (1) We propose Triangle Area Triple Loss Function (TATLF), which adopts Triangle Area (TA) as measurement of sample distance. TA comprehensively considers the absolute distance and included angle between samples, so that the trained model has better retrieval performance; (2) We combine CNN with Transformer to simultaneously extract local and edge features of reference images, which can effectively reduce the loss of reference images information. Specifically, CNN is adopted to extract local feature information of reference images. Transformer is used to pay attention to the edge feature information of reference images. Extensive experiments on two public datasets, Fashion200k and MIT-States, confirm the excellent performance of our proposed method.

show abstract

Segtransvae: Hybrid Cnn - Transformer with Regularization for Medical Image Segmentation

Cited by 18 publications

References 1 publication

Artificial Intelligence–Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review

Artificial Intelligence–Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review

CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer With Modality-Correlated Cross-Attention for Brain Tumor Segmentation

Composed query image retrieval based on triangle area triple loss function and combining CNN with transformer

Contact Info

Product

Resources

About