HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS

Huang, Chien-Hsiang; Wu, Hung-Yu; Lin, Youn-Long

doi:10.48550/arxiv.2101.07172

Cited by 49 publications

(93 citation statements)

References 27 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [39], only Kvasi dataset [33] is used and the evaluation metrics used include mDice, mIoU, recall and precision. From Table II, we can see that not only DS-TransUNet-L, but also DS-TransUNet-B outperforms the previous SOTA HarDNet-MSE [48] on all metrics. Specifically, DS-TransUNet-L achieves a mDice of 0.913, mIoU of 0.859, recall of 0.936 and precision of 0.916 with an improvement of 0.9%, 1.1%, 1.3% and 0.9%.…”

Section: A Comparison With State-of-the-art Methodsmentioning

confidence: 89%

DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation

Lin¹,

Chen²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automatic medical image segmentation has made great progress benefit from the development of deep learning. However, most existing methods are based on convolutional neural networks (CNNs), which fail to build long-range dependencies and global context connections due to the limitation of receptive field in convolution operation. Inspired by the success of Transformer whose self-attention mechanism has the powerful abilities of modeling the long-range contextual information, some researchers have expended considerable efforts in designing the robust variants of Transformer-based U-Net. Moreover, the patch division used in vision transformers usually ignores the pixellevel intrinsic structural features inside each patch. To alleviate these problems, in this paper, we propose a novel deep medical image segmentation framework called Dual Swin Transformer U-Net (DS-TransUNet), which might be the first attempt to concurrently incorporate the advantages of hierarchical Swin Transformer into both encoder and decoder of the standard Ushaped architecture to enhance the semantic segmentation quality of varying medical images. Unlike many prior Transformerbased solutions, the proposed DS-TransUNet first adopts dualscale encoder subnetworks based on Swin Transformer to extract the coarse and fine-grained feature representations of different semantic scales. As the core component for our DS-TransUNet, a well-designed Transformer Interactive Fusion (TIF) module is proposed to effectively establish global dependencies between features of different scales through the self-attention mechanism, in order to make full use of these obtained multi-scale feature representations. Furthermore, we also introduce the Swin Transformer block into decoder to further explore the longrange contextual information during the up-sampling process. Extensive experiments across four typical tasks for medical image segmentation demonstrate the effectiveness of DS-TransUNet, and show that our approach significantly outperforms the stateof-the-art methods.

show abstract

Section: A Comparison With State-of-the-art Methodsmentioning

confidence: 89%

DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation

Lin¹,

Chen²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Further, Tomar et al [32] designed a dual decoder attention network based on ResUNet++ for polyp segmentation. More recently, MSEG [33] improved the PraNet and proposed a simple encoder-decoder structure. Specifically, they used Hardnet [34] to replace the original backbone network Res2Net50 backbone network and removed the attention mechanism to achieve faster and more accurate polyp segmentation.…”

Section: A Polyp Segmentationmentioning

confidence: 99%

“…We implement our Polyp-PVT with the PyTorch framework and use a Tesla P100 to accelerate the calculations. Considering the differences in the sizes of each polyp image, we adopt a multi-scale strategy [5], [33] in the training stage. The hyperparameter details are as follows.…”

Section: G Implementation Detailsmentioning

confidence: 99%

“…Models. We collect several open source models from the field of polyp segmentation, for a total of nine comparative models, including U-Net [4], UNet++ [22], PraNet [5], SFA [80], MSEG [33], ACSNet [47], DCRNet [36], EU-Net [50] and SANet [9]. For fair comparison, we use their open source codes to evaluate on the same training and testing sets.…”

Section: B Datasets and Compared Modelsmentioning

confidence: 99%

“…4, where our model shows a steady improvement on both ColonDB and ETIS. Endoscene [14] Model mDic mIoU F w β Sα mE ξ maxE ξ MAE U-Net [4] 0.710 0.627 0.684 0.843 0.847 0.875 0.022 UNet++ [22] 0.707 0.624 0.687 0.839 0.834 0.898 0.018 SFA [80] 0.467 0.329 0.341 0.640 0.644 0.817 0.065 MSEG [33] 0.874 0.804 0.852 0.924 0.948 0.957 0.009 ACSNet [47] 0.573 0.519 0.547 0.703 0.698 0.941 0.235 DCRNet [36] 0.856 0.788 0.830 0.921 0.943 0.960 0.010 PraNet [5] 0.871 0.797 0.843 0.925 0.950 0.972 0.010 EU-Net [50] 0.837 0.765 0.805 0.904 0.919 0.933 0.015 SANet [9] 0.888 0.815 0.859 0.928 0.962 0.972 0.008 Polyp-PVT (Ours) 0.900 0.833 0.884 0.935 0.973 0.981 0.007…”

Section: Quantitative Analysis Of Generalization Abilitymentioning

confidence: 99%

See 2 more Smart Citations

Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers

Dong¹,

Wang²,

Fan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Most polyp segmentation methods use CNNs as their backbone, leading to two key issues when exchanging information between the encoder and decoder: 1) taking into account the differences in contribution between different-level features; and 2) designing effective mechanism for fusing these features. Different from existing CNN-based methods, we adopt a transformer encoder, which learns more powerful and robust representations. In addition, considering the image acquisition influence and elusive properties of polyps, we introduce three novel modules, including a cascaded fusion module (CFM), a camouflage identification module (CIM), a and similarity aggregation module (SAM). Among these, the CFM is used to collect the semantic and location information of polyps from high-level features, while the CIM is applied to capture polyp information disguised in low-level features. With the help of the SAM, we extend the pixel features of the polyp area with high-level semantic position information to the entire polyp area, thereby effectively fusing cross-level features. The proposed model, named Polyp-PVT, effectively suppresses noises in the features and significantly improves their expressive capabilities. Extensive experiments on five widely adopted datasets show that the proposed model is more robust to various challenging situations (e.g., appearance changes, small objects) than existing methods, and achieves the new state-of-the-art performance. The proposed model is available at https://github.com/DengPingFan/Polyp-PVT.

show abstract

Multi‐scale nested UNet with transformer for colorectal polyp segmentation

Wang,

Liu,

et al. 2024

J Applied Clin Med Phys

View full text Add to dashboard Cite

BackgroundPolyp detection and localization are essential tasks for colonoscopy. U‐shape network based convolutional neural networks have achieved remarkable segmentation performance for biomedical images, but lack of long‐range dependencies modeling limits their receptive fields.PurposeOur goal was to develop and test a novel architecture for polyp segmentation, which takes advantage of learning local information with long‐range dependencies modeling.MethodsA novel architecture combining with multi‐scale nested UNet structure integrated transformer for polyp segmentation was developed. The proposed network takes advantage of both CNN and transformer to extract distinct feature information. The transformer layer is embedded between the encoder and decoder of a U‐shape net to learn explicit global context and long‐range semantic information. To address the challenging of variant polyp sizes, a MSFF unit was proposed to fuse features with multiple resolution.ResultsFour public datasets and one in‐house dataset were used to train and test the model performance. Ablation study was also conducted to verify each component of the model. For dataset Kvasir‐SEG and CVC‐ClinicDB, the proposed model achieved mean dice score of 0.942 and 0.950 respectively, which were more accurate than the other methods. To show the generalization of different methods, we processed two cross dataset validations, the proposed model achieved the highest mean dice score. The results demonstrate that the proposed network has powerful learning and generalization capability, significantly improving segmentation accuracy and outperforming state‐of‐the‐art methods.ConclusionsThe proposed model produced more accurate polyp segmentation than current methods on four different public and one in‐house datasets. Its capability of polyps segmentation in different sizes shows the potential clinical application

show abstract

HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS

Abstract: Figure 1: Mean Dice accuracy vs frame rate running on a GeForce RTX 2080 Ti GPU as reported in [20](blue) and [13](orange). HarDNet-MSEG is faster and more accurate than the SOTA (U-Net[ResNet34] and PraNet).

Cited by 49 publications

References 27 publications

DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation

DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation

Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers

Multi‐scale nested UNet with transformer for colorectal polyp segmentation

Contact Info

Product

Resources

About