Towards Efficient Medical Image Segmentation Via Boundary-Guided Knowledge Distillation

Wen, Yang; Chen, Leiting; Xi, Shuo; Deng, Yu; Tang, Ximan; Zhou, Chuan

doi:10.1109/icme51207.2021.9428395

Cited by 6 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As for classification [168], constraints on intermediate multiscale features arising from both teacher and student were therefore integrated [33] with importance maps distillation modules able to encode feature maps into a transformable form to deal with the diversity of feature sizes between teacher and student models. Other constraints were proposed in the specific context of KD, such as boundary-guided [171], region affinity [33], class-similarity [172], anatomical knowledge [173], or holistic distillation [174] in order to align high-order relations between what both teacher and student generated.…”

Section: E Knowledge Distillationmentioning

confidence: 99%

Current and Emerging Trends in Medical Image Segmentation With Deep Learning

Conze

Andrade-Miranda

Singh

et al. 2023

IEEE Trans. Radiat. Plasma Med. Sci.

View full text Add to dashboard Cite

In recent years, the segmentation of anatomical or pathological structures using deep learning has experienced a widespread interest in medical image analysis. Remarkably successful performance has been reported in many imaging modalities and for a variety of clinical contexts to support clinicians in computer-assisted diagnosis, therapy, or surgical planning purposes. However, despite the increasing amount of medical image segmentation challenges, there remains little consensus on which methodology performs best. Therefore, we examine in this article the numerous developments and breakthroughs brought since the rise of U-Net-inspired architectures. Especially, we focus on the technical challenges and emerging trends that the community is now focusing on, including conditional generative adversarial and cascaded networks, medical Transformers, contrastive learning, knowledge distillation, active learning, prior knowledge embedding, cross-modality learning, multistructure analysis, federated learning, or semi-supervised and self-supervised paradigms. We also suggest possible avenues to be further investigated in future research efforts.

show abstract

Section: E Knowledge Distillationmentioning

confidence: 99%

Current and Emerging Trends in Medical Image Segmentation With Deep Learning

Conze

Andrade-Miranda

Singh

et al. 2023

IEEE Trans. Radiat. Plasma Med. Sci.

View full text Add to dashboard Cite

show abstract

“…However, only a few researchers introduce knowledge distillation for efficient medical image segmentation. Wen et al [20] propose a boundary-guided knowledge distillation method which assists the student network to align organ boundary features in teacher network. Qin et al [21] design a region affinity distillation method, and integrate the importance map for efficient liver tumor segmentation.…”

Section: B Knowledge Distillationmentioning

confidence: 99%

“…Though the knowledge distillation has developed deeply in computer vision, only a few researchers try to explore knowledge distillation in the medical imaging community. Wen et al [20] propose a boundary-guided distillation method for organ segmentation. Qin et al [21] design a region affinity distillation method for efficient liver tumor segmentation.…”

Section: Introductionmentioning

confidence: 99%

Graph Flow: Cross-layer Graph Flow Distillation for Dual Efficient Medical Image Segmentation

Zou¹,

Sun²

2022

Preprint

View full text Add to dashboard Cite

With the development of deep convolutional neural networks, medical image segmentation has achieved a series of breakthroughs in recent years. However, the higher-performance convolutional neural networks always mean numerous parameters and high computation costs, which will hinder the applications in clinical scenarios. Meanwhile, the scarceness of large-scale annotated medical image datasets further impedes the application of high-performance networks. To tackle these problems, we propose Graph Flow, a comprehensive knowledge distillation framework, for both network-efficiency and annotation-efficiency medical image segmentation. Specifically, our core Graph Flow Distillation transfer the essence of cross-layer variations from a well-trained cumbersome teacher network to a non-trained compact student network. In addition, an unsupervised Paraphraser Module is designed to purify the knowledge of the teacher network, which is also beneficial for the stabilization of training procedure. Furthermore, we build a unified distillation framework by integrating the adversarial distillation and the vanilla logits distillation, which can further refine the final predictions of the compact network. Extensive experiments conducted on Gastric Cancer Segmentation Dataset and Synapse Multi-organ Segmentation Dataset demonstrate the prominent ability of our method which achieves state-of-the-art performance on these differentmodality and multi-category medical image datasets. Moreover, we demonstrate the effectiveness of our Graph Flow through a new semi-supervised paradigm for dual efficient medical image segmentation. Our code will be available at Graph Flow .

show abstract