The Lighter the Better: Rethinking Transformers in Medical Image Segmentation Through Adaptive Pruning

Lin, Xian; Yu, Li; Cheng, Kwang-Ting; Yan, Zengqiang

doi:10.1109/tmi.2023.3247814

Cited by 11 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The quantitative results of existing methods are reported by Lin et al (2022) ; Wu et al (2022) ; Reza et al (2022) ( Table 4 ). Our method achieves the highest scores in all metrics except SE with a slight decrease.…”

Section: Methodsmentioning

confidence: 99%

Robust fusion for skin lesion segmentation of dermoscopic images

Guo

Fang

Wang

et al. 2023

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

Robust skin lesion segmentation of dermoscopic images is still very difficult. Recent methods often take the combinations of CNN and Transformer for feature abstraction and multi-scale features for further classification. Both types of combination in general rely on some forms of feature fusion. This paper considers these fusions from two novel points of view. For abstraction, Transformer is viewed as the affinity exploration of different patch tokens and can be applied to attend CNN features in multiple scales. Consequently, a new fusion module, the Attention-based Transformer-And-CNN fusion module (ATAC), is proposed. ATAC augments the CNN features with more global contexts. For further classification, adaptively combining the information from multiple scales according to their contributions to object recognition is expected. Accordingly, a new fusion module, the GAting-based Multi-Scale fusion module (GAMS), is also introduced, which adaptively weights the information from multiple scales by the light-weighted gating mechanism. Combining ATAC and GAMS leads to a new encoder-decoder-based framework. In this method, ATAC acts as an encoder block to progressively abstract strong CNN features with rich global contexts attended by long-range relations, while GAMS works as an enhancement of the decoder to generate the discriminative features through adaptive fusion of multi-scale ones. This framework is especially good at lesions of varying sizes and shapes and of low contrasts and its performances are demonstrated with extensive experiments on public skin lesion segmentation datasets.

show abstract

Section: Methodsmentioning

confidence: 99%

Robust fusion for skin lesion segmentation of dermoscopic images

Guo

Fang

Wang

et al. 2023

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

show abstract

“…Comparison Methods. SOTA 2D CNN-/Transformerbased task-specific approaches are adopted for evaluation, including R50 U-Net (Chen et al 2021), R50 Att-UNet (Chen et al 2021), UNet (Ronneberger, Fischer, and Brox 2015), Att-UNet (Schlemper et al 2019), SwinUNet (Cao et al 2022), TransClaw U-Net (Yao et al 2022), LeVit-UNet-384 (Xu et al 2021), MT-UNet (Wang et al 2022), MISSFormer (Huang et al 2022), CA-GANformer (You et al 2022), and APFormer (Lin et al 2023b).…”

Section: Evaluation On Btcvmentioning

confidence: 99%

“…Therefore, developing CNN-Transformer hybrid architectures has been extensively studied for medical image segmentation. Unfortunately, one underlying but rarely-explored issue in existing CNN-Transformer hybrid frameworks is attention collapse (Lin et al 2023a) where all patches/tokens share the same dependency distribution. In other words, transformer becomes a bypass module, completely failing to extract meaningful global features, as illustrated in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DTMFormer: Dynamic Token Merging for Boosting Transformer-Based Medical Image Segmentation

Wang,

Lin,

et al. 2024

AAAI

View full text Add to dashboard Cite

Despite the great potential in capturing long-range dependency, one rarely-explored underlying issue of transformer in medical image segmentation is attention collapse, making it often degenerate into a bypass module in CNN-Transformer hybrid architectures. This is due to the high computational complexity of vision transformers requiring extensive training data while well-annotated medical image data is relatively limited, resulting in poor convergence. In this paper, we propose a plug-n-play transformer block with dynamic token merging, named DTMFormer, to avoid building long-range dependency on redundant and duplicated tokens and thus pursue better convergence. Specifically, DTMFormer consists of an attention-guided token merging (ATM) module to adaptively cluster tokens into fewer semantic tokens based on feature and dependency similarity and a light token reconstruction module to fuse ordinary and semantic tokens. In this way, as self-attention in ATM is calculated based on fewer tokens, DTMFormer is of lower complexity and more friendly to converge. Extensive experiments on publicly-available datasets demonstrate the effectiveness of DTMFormer working as a plug-n-play module for simultaneous complexity reduction and performance improvement. We believe it will inspire future work on rethinking transformers in medical image segmentation. Code: https://github.com/iam-nacl/DTMFormer.

show abstract

“…27,28 The prevailing network design strategy involves embedding Transformers into U-Net frameworks, exemplified by architectures like TransUnet, 29 Swin-Unet, 30 H2Former, 31 UCTransNet, 32 and others. 33,34 Nevertheless, when addressing the challenge of inconspicuous target regions arising from noise and artifacts in ultrasound images, the aforementioned methods are primarily addressed through preprocessing techniques like normalization and the establishment of pixel enhancement thresholds. However, such processing often leads to the loss of image edges and internal information, presenting challenges to the accurate segmentation of the target region.…”

mentioning

confidence: 99%

A target region extraction method for ultrasound medical images based on improved PRIDNet and UCTransNet

Zhai,

Tian,

et al. 2024

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Computer‐aided diagnosis is pivotal in augmenting the diagnostic efficiency of ultrasound images. Nonetheless, the substantial presence of noise and artifacts in ultrasound images presents a challenge to the precise segmentation of the target region. To highlight and analyze the diagnostic information such as tissues, organs, and lesions in ultrasound medical images more accurately, a target region extraction method combining the improved PRIDNet and UCTransNet for ultrasound medical images is proposed. The method sequentially enhances and segments the target region of the image, thereby solving the problem of inconspicuous target region caused by noise and artifacts in ultrasound images. There are three key characteristics: (i) The feature extraction part of the enhancement network (improved PRIDNet) is redesigned for speckle noise to improve the network's ability to extract information and highlight the feature information of the target region in ultrasound images. (ii) The segmentation network with the addition of the underlying information on UCTransNet would effectively improves Channel‐wise Cross fusion Transformer (CCT) and decoder feature fusion capability. (iii) By combining the enhancement network with the segmentation network, we can further improve the segmentation accuracy of the target region in the presence of noise interference. The experiments conducted on both UFSU that we prepared and on some public datasets including BUSI, FHC, and CT2US have demonstrated that The proposed method attains MIoU, DSC, Acc, and HD of 96.34%, 98.12%, 99.35%, and 8.64 in CT2US, respectively. The method significantly surpasses those certain state‐of‐the‐art methods, demonstrating its potential to offer valuable guidance for clinical treatment. The code will be publicly released at https://github.com/425877/Target-Region-Extraction-Method-for-Ultrasound-Medical-Images.

show abstract

The Lighter the Better: Rethinking Transformers in Medical Image Segmentation Through Adaptive Pruning

Cited by 11 publications

References 21 publications

Robust fusion for skin lesion segmentation of dermoscopic images

Robust fusion for skin lesion segmentation of dermoscopic images

DTMFormer: Dynamic Token Merging for Boosting Transformer-Based Medical Image Segmentation

A target region extraction method for ultrasound medical images based on improved PRIDNet and UCTransNet

Contact Info

Product

Resources

About