TransConver: transformer and convolution parallel network for developing automatic brain tumor segmentation in MRI images

Liang, Junjie; Yang, Cihui; Zeng, Mengjie; Wang, Xixi

doi:10.21037/qims-21-919

Cited by 26 publications

(32 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TransFuse (Zhang et al, 2021 ) combines transformers and CNNs in a parallel style to capture both global dependency and low-level spatial details efficiently in a much shallower manner for medical image segmentations. Liang et al ( 2022 ) proposed transconver with a parallel module named transformer-convolution inception which extracts local and global information via convolution blocks and transformer blocks, respectively. TransMed (Dai et al, 2021 ) was proposed for multi-modal medical image classification which combines the advantages of CNN and transformer to extract low-level features of images efficiently and establish long-range dependencies between modalities.…”

Section: Related Workmentioning

confidence: 99%

O-Net: A Novel Framework With Deep Fusion of CNN and Transformer for Simultaneous Segmentation and Classification

et al. 2022

View full text Add to dashboard Cite

The application of deep learning in the medical field has continuously made huge breakthroughs in recent years. Based on convolutional neural network (CNN), the U-Net framework has become the benchmark of the medical image segmentation task. However, this framework cannot fully learn global information and remote semantic information. The transformer structure has been demonstrated to capture global information relatively better than the U-Net, but the ability to learn local information is not as good as CNN. Therefore, we propose a novel network referred to as the O-Net, which combines the advantages of CNN and transformer to fully use both the global and the local information for improving medical image segmentation and classification. In the encoder part of our proposed O-Net framework, we combine the CNN and the Swin Transformer to acquire both global and local contextual features. In the decoder part, the results of the Swin Transformer and the CNN blocks are fused to get the final results. We have evaluated the proposed network on the synapse multi-organ CT dataset and the ISIC 2017 challenge dataset for the segmentation task. The classification network is simultaneously trained by using the encoder weights of the segmentation network. The experimental results show that our proposed O-Net achieves superior segmentation performance than state-of-the-art approaches, and the segmentation results are beneficial for improving the accuracy of the classification task. The codes and models of this study are available at https://github.com/ortonwang/O-Net.

show abstract

Section: Related Workmentioning

confidence: 99%

O-Net: A Novel Framework With Deep Fusion of CNN and Transformer for Simultaneous Segmentation and Classification

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The segmentation loss is computed using the label and the consistency loss is obtained between two predictions. Liang and colleagues [120] developed a network named TransConver to segment MRI images using multiple datasets [121,122,123]. The encoder is composed of a convolution block, three TC-inception blocks, and three downsample operations.…”

Section: Segmentationmentioning

confidence: 99%

“…The proposed model is composed of a down-sample part, an up-sample part, and a connection part. The UTNet [110] 2021 MRI left ventricle, right ventricle, left ventricular myocardium [111] MRA-TUNet [112] 2022 MRI left ventricle, right ventricle, left ventricular myocardium, left atrium cardiac disease [113], atrial fibrillation [114] HybridCTrm [115] 2021 MRI brain [116], neurodevelopmental disorders [117] consistency-based co-segmentation [118] 2021 MRI right ventricle [119] TransConver [120] 2022 MRI brain brain tumor [121,122,123] UTransNet [124] 2022 MRI brain stroke [129] TransBTS [125] 2021 MRI brain brain tumor [121,122,123] METrans [126] 2022 MRI brain stroke [130], ischemic stroke lesion [131], schemic stroke lesion [132] SwinBTS [127] 2022 MRI brain brain tumor [121,123,133,134] BTSwin-Unet [128] 2022 MRI brain brain tumor [121,122] CVT-Vnet [135] 2022 CT head, neck organs at risk [136] CoTr [137] 2021 CT abdomen colorectal cancer, ventral hernia [138] transformer-UNet [139] 2021 CT lung [140] AFTer-UNet [141] 2022 CT abdomen, thorax [142], organs at risk…”

Section: Segmentationmentioning

confidence: 99%

Recent Progress in Transformer-based Medical Image Analysis

Liu¹,

Shen²

2022

Preprint

View full text Add to dashboard Cite

The transformer has dominated the natural language processing (NLP) field for a long time. Recently, the transformer-based method has been adopted into the computer vision (CV) field and shows promising results. As an important branch of the CV field, medical image analysis joins the wave of the transformer-based method rightfully. In this review, we illustrate the principle of the attention mechanism, and the detailed structures of the transformer, and depict how the transformer is adopted into medical image analysis. We organize the transformer-based medical image analysis applications in a sequence of different tasks, including classification, segmentation, synthesis, registration, localization, detection, captioning, and denoising. For the mainstream classification and segmentation tasks, we further divided the corresponding works based on different medical imaging modalities. The datasets corresponding to the related works are also organized. We include thirteen modalities and more than twenty objects in our work.

show abstract

“…Currently, deep learning (DL), represented by convolutional neural networks (CNNs) and transformers, has achieved enormous success in the field of medical image segmentation. CNN is a classical DL architecture that extracts hierarchical features from an image via convolution operation to identify the region of interest pixel by pixel, and is widely used in the segmentations on MR images of brain tumor (12), breast tumor (13), colorectal tumor (14) and etc. The transformer is a novel DL architecture which had first attracted much attention due to its ability in natural language processing (NLP), and has recently demonstrated promising results on tasks involving computer vision.…”

Section: Introductionmentioning

confidence: 99%

Deep learning-based segmentation of epithelial ovarian cancer on T2-weighted magnetic resonance images

Hu¹,

Jian²,

Li³

et al. 2023

Quant Imaging Med Surg

View full text Add to dashboard Cite

Background: Epithelial ovarian cancer (EOC) segmentation is an indispensable step in assessing the extent of disease and guiding the treatment plan that follows. Currently, manual segmentation is the most commonly used method, despite it being tedious, time-consuming and subject to inter-and intra-observer variability. This study aims to assess the feasibility of deep learning methods in the automatic segmentation of EOC on T2-weighted magnetic resonance images. Methods: A total of 339 EOC patients from eight different clinical centers were enrolled and divided into 4 groups: training set (n=154), validation set (n=25), internal test set (n=50) and external test set (n=110). Six common-used algorithms, including convolutional neural networks (CNNs) (U-Net, DeepLabv3, U-Net++ and PSPNet) and transformers (TransUnet and Swin-Unet), were used to conduct automatic segmentations. The performances of these automatic segmentation methods were evaluated by means of dice similarity coefficient (DSC), Hausdorff distance (HD), average symmetric surface distance (ASSD), precision and recall.Results: All the results look promising, which demonstrates the feasibility of using deep learning for EOC segmentation. Overall, CNNs and transformers showed similar performances in both internal and external test sets. Among all the models, U-Net++ performed best with a DSC, HD, ASSD, precision and

show abstract

TransConver: transformer and convolution parallel network for developing automatic brain tumor segmentation in MRI images

Cited by 26 publications

References 15 publications

O-Net: A Novel Framework With Deep Fusion of CNN and Transformer for Simultaneous Segmentation and Classification

O-Net: A Novel Framework With Deep Fusion of CNN and Transformer for Simultaneous Segmentation and Classification

Recent Progress in Transformer-based Medical Image Analysis

Deep learning-based segmentation of epithelial ovarian cancer on T2-weighted magnetic resonance images

Contact Info

Product

Resources

About