Multi-label Whole Heart Segmentation Using CNNs and Anatomical Label Configurations

Payer, Christian; Štern, Darko; Bischof, Horst; Urschler, Martin

doi:10.1007/978-3-319-75541-0_20

Cited by 124 publications

(130 citation statements)

References 10 publications

Supporting

Mentioning

121

Contrasting

Unclassified

Order By: Relevance

“…Both [9] and [19] utilize independent encoders/decoders for each modality, while we just use modalityspecific BN layers resulting in a more compact model. By further leveraging KD-loss as sort of cross-modality transductive bias, the segmentation performance is boosted to overall Dice of 88.8% (specifically, 91.7% on CT and 86.0% on MRI), exceeding our own implemented "Indivudial" training as well as the MICCAI-MMWHS challenge winner Payer et al [38] (overall Dice of 85.5%) which used single model learning.…”

Section: Segmentation Results and Comparison With State-of-the-artsmentioning

confidence: 97%

See 1 more Smart Citation

Unpaired Multi-Modal Segmentation via Knowledge Distillation

Dou

Liu

Heng

et al. 2020

IEEE Trans. Med. Imaging

151

100

View full text Add to dashboard Cite

Multi-modal learning is typically performed with network architectures containing modality-specific layers and shared layers, utilizing co-registered images of different modalities. We propose a novel learning scheme for unpaired crossmodality image segmentation, with a highly compact architecture achieving superior segmentation accuracy. In our method, we heavily reuse network parameters, by sharing all convolutional kernels across CT and MRI, and only employ modality-specific internal normalization layers which compute respective statistics.To effectively train such a highly compact model, we introduce a novel loss term inspired by knowledge distillation, by explicitly constraining the KL-divergence of our derived prediction distributions between modalities. We have extensively validated our approach on two multi-class segmentation problems: i) cardiac structure segmentation, and ii) abdominal organ segmentation. Different network settings, i.e., 2D dilated network and 3D Unet, are utilized to investigate our method's general efficacy. Experimental results on both tasks demonstrate that our novel multi-modal learning scheme consistently outperforms singlemodal training and previous multi-modal approaches.

show abstract

Section: Segmentation Results and Comparison With State-of-the-artsmentioning

confidence: 97%

“…We compare the performance of different multi-modal learning methods, including two state-of-the-art approaches [9], [15]. We also refer to the available winning performance of the challenge [36], [38] to demonstrate effectiveness of multi-modal learning.…”

Section: Segmentation Results and Comparison With State-of-the-artsmentioning

confidence: 99%

Unpaired Multi-Modal Segmentation via Knowledge Distillation

Dou

Liu

Heng

et al. 2020

IEEE Trans. Med. Imaging

151

100

View full text Add to dashboard Cite

show abstract

“…Our current network architectures follow the practice of CycleGAN [8] by using the Resnet blocks for the generator and decoder, and follow the previous cross-modality adaptation work [7] for the configurations of the segmentation model. To validate the effectiveness of our segmentation backbone, we compare the supervised training performance of our segmentation model on cardiac dataset with Payer et al [54], which obtained the first ranking in the MMWHS Challenge 2017. Table III shows that our model can achieve comparable performance to Payer et al But unlike [7], [44] which prefers network architectures without skip connections, we consider that other network architectures could also be used in our framework, such as the Unet model [1], which is the most common networks for medical image segmentation.…”

Section: Discussionmentioning

confidence: 99%

Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation

Chen

Dou

Chen

et al. 2020

IEEE Trans. Med. Imaging

314

185

View full text Add to dashboard Cite

Unsupervised domain adaptation has increasingly gained interest in medical image computing, aiming to tackle the performance degradation of deep neural networks when being deployed to unseen data with heterogeneous characteristics. In this work, we present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA), to effectively adapt a segmentation network to an unlabeled target domain. Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives. In particular, we simultaneously transform the appearance of images across domains and enhance domain-invariance of the extracted features by leveraging adversarial learning in multiple aspects and with a deeply supervised mechanism. The feature encoder is shared between both adaptive perspectives to leverage their mutual benefits via end-to-end learning. We have extensively evaluated our method with cardiac substructure segmentation and abdominal multi-organ segmentation for bidirectional crossmodality adaptation between MRI and CT images. Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images, and outperforms the state-of-the-art domain adaptation approaches by a large margin.

show abstract

“…Payer et al implemented a U-Net for substructure segmentation and obtained a DSC of 94% in the aorta as compared to ground truth. 20 Various DNNs have been applied to medical image segmentation, 19 specifically for cardiac substructure segmentation. These include deep convolutional neural networks (CNNs) with adaptive fusion 21 or multi-stage 20 strategies, as well as generative adversarial networks (GANs) 22 .…”

Section: Introductionmentioning

confidence: 99%

Cardiac substructure segmentation with deep learning for improved cardiac sparing

et al. 2019

View full text Add to dashboard Cite

Purpose Radiation dose to cardiac substructures is related to radiation‐induced heart disease. However, substructures are not considered in radiation therapy planning (RTP) due to poor visualization on CT. Therefore, we developed a novel deep learning (DL) pipeline leveraging MRI’s soft tissue contrast coupled with CT for state‐of‐the‐art cardiac substructure segmentation requiring a single, non‐contrast CT input. Materials/methods Thirty‐two left‐sided whole‐breast cancer patients underwent cardiac T2 MRI and CT‐simulation. A rigid cardiac‐confined MR/CT registration enabled ground truth delineations of 12 substructures (chambers, great vessels (GVs), coronary arteries (CAs), etc.). Paired MRI/CT data (25 patients) were placed into separate image channels to train a three‐dimensional (3D) neural network using the entire 3D image. Deep supervision and a Dice‐weighted multi‐class loss function were applied. Results were assessed pre/post augmentation and post‐processing (3D conditional random field (CRF)). Results for 11 test CTs (seven unique patients) were compared to ground truth and a multi‐atlas method (MA) via Dice similarity coefficient (DSC), mean distance to agreement (MDA), and Wilcoxon signed‐ranks tests. Three physicians evaluated clinical acceptance via consensus scoring (5‐point scale). Results The model stabilized in ~19 h (200 epochs, training error <0.001). Augmentation and CRF increased DSC 5.0 ± 7.9% and 1.2 ± 2.5%, across substructures, respectively. DL provided accurate segmentations for chambers (DSC = 0.88 ± 0.03), GVs (DSC = 0.85 ± 0.03), and pulmonary veins (DSC = 0.77 ± 0.04). Combined DSC for CAs was 0.50 ± 0.14. MDA across substructures was <2.0 mm (GV MDA = 1.24 ± 0.31 mm). No substructures had statistical volume differences (P > 0.05) to ground truth. In four cases, DL yielded left main CA contours, whereas MA segmentation failed, and provided improved consensus scores in 44/60 comparisons to MA. DL provided clinically acceptable segmentations for all graded patients for 3/4 chambers. DL contour generation took ~14 s per patient. Conclusions These promising results suggest DL poses major efficiency and accuracy gains for cardiac substructure segmentation offering high potential for rapid implementation into RTP for improved cardiac sparing.

show abstract

Multi-label Whole Heart Segmentation Using CNNs and Anatomical Label Configurations

Cited by 124 publications

References 10 publications

Unpaired Multi-Modal Segmentation via Knowledge Distillation

Unpaired Multi-Modal Segmentation via Knowledge Distillation

Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation

Cardiac substructure segmentation with deep learning for improved cardiac sparing

Contact Info

Product

Resources

About