IB-TransUNet: Combining Information Bottleneck and Transformer for Medical Image Segmentation

Li, Guangju; Jin, Dehu; Yu, Qi; Meng, Qiang

doi:10.1016/j.jksuci.2023.02.012

Cited by 9 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A deep supervised transformer full-resolution residual network was presented in [ 37 ]. Its feature fusion is better and suppresses irrelevant features, while the deep supervision mechanism reduces the gradient vanishing problem.…”

Section: Organsmentioning

confidence: 99%

Ultrasound Image Analysis with Vision Transformers—Review

Vafaeezadeh,

Behnam,

Gifani

2024

Diagnostics

View full text Add to dashboard Cite

Ultrasound (US) has become a widely used imaging modality in clinical practice, characterized by its rapidly evolving technology, advantages, and unique challenges, such as a low imaging quality and high variability. There is a need to develop advanced automatic US image analysis methods to enhance its diagnostic accuracy and objectivity. Vision transformers, a recent innovation in machine learning, have demonstrated significant potential in various research fields, including general image analysis and computer vision, due to their capacity to process large datasets and learn complex patterns. Their suitability for automatic US image analysis tasks, such as classification, detection, and segmentation, has been recognized. This review provides an introduction to vision transformers and discusses their applications in specific US image analysis tasks, while also addressing the open challenges and potential future trends in their application in medical US image analysis. Vision transformers have shown promise in enhancing the accuracy and efficiency of ultrasound image analysis and are expected to play an increasingly important role in the diagnosis and treatment of medical conditions using ultrasound imaging as technology progresses.

show abstract

Section: Organsmentioning

confidence: 99%

Ultrasound Image Analysis with Vision Transformers—Review

Vafaeezadeh,

Behnam,

Gifani

2024

Diagnostics

View full text Add to dashboard Cite

show abstract

“…The BGRD-TransUNet model is an improvement upon the TransUNet model. Currently, some models based on Tran-sUNet, such as DA-TransUNet [28] and IB-TransUNet [10], introduce bottleneck modules between the CNN and Transformer to optimize the feature maps fed into the Transformer. However, the CNN components of these models still utilize the original ResNet50 [13] structure of TransUNet without modification.…”

Section: A Overall Structurementioning

confidence: 99%

“…However, due to the inherent limitations of CNN operations, U-Net often struggles to capture global contextual information, [9]. This limitation arises because the size of the convolutional kernel in CNN determines its local perception ability, restricting a single convolutional kernel to focus on a limited local region, [10]. While CNN models can gradually expand the receptive field of convolutional kernels by stacking numerous convolutional layers (which allows higher-level convolutional kernels to focus on global features), lower and intermediate-level convolutional kernels remain constrained and can only focus on local regions.…”

Section: Introductionmentioning

confidence: 99%

BGRD-TransUNet: A Novel TransUNet-Based Model for Ultrasound Breast Lesion Segmentation

Ji,

Sun,

Yuan

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Breast UltraSound (BUS) imaging is a commonly used diagnostic tool in the field of counter fighting breast diseases, especially for early detection and diagnosis of breast cancer. Due to the inherent characteristics of ultrasound images such as blurry boundaries and diverse tumor morphologies, it is challenging for doctors to manually segment breast tumors. In recent years, the Convolutional Neural Network (CNN) technology has been widely applied to automatically segment BUS images. However, due to the inherent limitations of CNNs in capturing global contextual information, it is difficult to capture the full context. To address this issue, the paper proposes a novel BGRD-TransUNet model for breast lesion segmentation, based on TransUNet. The proposed model, first, replaces the original ResNet50 backbone network of TransUNet with DenseNet121 for initial feature extraction. Next, newly designed Residual Multi-Scale Feature Modules (RMSFMs) are employed to extract features from various layers of DenseNet121, thus capturing richer features within specific layers. Thirdly, a Boundary Guidance (BG) network is added to enhance the contour information of BUS images. Additionally, newly designed Boundary Attentional Feature Fusion Modules (BAFFMs) are used to integrate edge information and features extracted through RMSFMs. Finally, newly designed Parallel Channel and Spatial Attention Modules (PCSAMs) are used to refine feature extraction using channel and spatial attention. An extensive experimental testing performed on two public datasets demonstrates that the proposed BGRD-TransUNet model outperforms all state-of-the-art medical image segmentation models, participating in the experiments, according to all evaluation metrics used (except for few separate cases), including the two most important and widely used metrics in the field of medical image segmentation, namely the Intersection over Union (IoU) and Dice Similarity Coefficient (DSC). More specifically, on the BUSI dataset and dataset B, BGRD-TransUNet achieves IoU values of 76.77% and 86.61%, and DSC values of 85.08% and 92.47%, respectively, which are higher by 7.27 and 3.64, and 5.81 and 2.54 percentage points, than the corresponding values achieved by the baseline (TransUNet).

show abstract

“…These transformers divide images into patches and apply the attention mechanism between each set of patches. Yet, since there is no natural way of reducing or expanding the dimensionality of the data, the ViT has been used to dimensionality reduction tasks only to a limited degree [33][34][35][36] . Importantly, current ViTs have not been integrated with increasing numbers of channels in convolutional layers to represent increasingly complicated features.…”

Section: Transformer-based Dimensionality Reductionmentioning

confidence: 99%

The Deep Latent Space Particle Filter for Real-Time Data Assimilation with Uncertainty Quantification

Mücke,

Bohté,

Oosterlee

2024

Preprint

View full text Add to dashboard Cite

In Data Assimilation, observations are fused with simulations to obtain an accurate estimate of the state and parameters for a given physical system. Combining data with a model, however, while accurately estimating uncertainty, is computationally expensive and infeasible to run in real-time for complex systems. Here, we present a novel particle filter methodology, the Deep Latent Space Particle filter or D-LSPF, that uses neural network-based surrogate models to overcome this computational challenge. The D-LSPF enables filtering in the low-dimensional latent space obtained using Wasserstein AEs with modified vision transformer layers for dimensionality reduction and transformers for parameterized latent space time stepping. As we demonstrate on three test cases, including leak localization in multi-phase pipe flow and seabed identification for fully nonlinear water waves, the D-LSPF runs orders of magnitude faster than a high-fidelity particle filter and 3-5 times faster than alternative methods while being up to an order of magnitude more accurate. The D-LSPF thus enables real-time data assimilation with uncertainty quantification for physical systems.

show abstract

IB-TransUNet: Combining Information Bottleneck and Transformer for Medical Image Segmentation

Cited by 9 publications

References 14 publications

Ultrasound Image Analysis with Vision Transformers—Review

Ultrasound Image Analysis with Vision Transformers—Review

BGRD-TransUNet: A Novel TransUNet-Based Model for Ultrasound Breast Lesion Segmentation

The Deep Latent Space Particle Filter for Real-Time Data Assimilation with Uncertainty Quantification

Contact Info

Product

Resources

About