SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth

Abstract

A key limitation of deep convolutional neural networks (DCNN) based image segmentation methods is the lack of generalizability. Manually traced training images are typically required when segmenting organs in a new imaging modality or from distinct disease cohort. The manual efforts can be alleviated if the manually traced images in one imaging modality (e.g., MRI) are able to train a segmentation network for another imaging modality (e.g., CT). In this paper, we propose an end-to-end synthetic segmentation network (SynSeg-Net) to train a segmentation network for a target imaging modality without having manual labels. SynSeg-Net is trained by using (1) unpaired intensity images from source and target modalities, and (2) manual labels only from source modality. SynSeg-Net is enabled by the recent advances of cycle generative adversarial networks (CycleGAN) and DCNN. We evaluate the performance of the SynSeg-Net on two experiments: (1) MRI to CT splenomegaly synthetic segmentation for abdominal images, and (2) CT to MRI total intracranial volume synthetic segmentation (TICV) for brain images. The proposed end-to-end approach achieved superior performance to two stage methods. Moreover, the SynSeg-Net achieved comparable performance to the traditional segmentation network using target modality labels in certain scenarios. The source code of SynSeg-Net is publicly available 2 .

Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation

“…To alleviate the burden of data annotation, some works focus on cross-modality image synthesis so that the segmentation of multiple modalities can be achieved with synthesized images and one-modality labels [51]- [53]. Recently, some works have explored the feasibility of cross-modality unsupervised domain adaptation to adapt deep models from the label-rich source modality to unlabeled target modality [7], [37], with good results reported. Our work proceeds along this promising direction, by demonstrating that without extra annotation effort, unsupervised domain adaptation can greatly reduce the performance degradation and for some tasks can even achieve very close segmentation performance to supervised training.…”

“…A higher Dice value and a lower ASD value indicate better segmentation results. The evaluation is performed on the subject-level segmentation volume to be consistent with the MMWHS and CHAOS challenges as well as previous works [37], [44].…”

“…For cross-modality adaptation, Jiang et al [36] first transform CT images to resemble MRI appearance using CycleGAN with tumor-aware loss, then the generated MRI images are combined with a few real MRI data for semi-supervised tumor segmentation. In [37] and [38], CycleGAN is combined with a segmentation network to compose an end-to-end framework. Compared to [37] and [38], a key characteristic of our approach is the shared encoder for both image transformation and segmentation task.…”

A multimodal computational pipeline for 3D histology of the human brain

“…Recent studies have proposed ROI-based approaches 79 to better understand the microscopic substrate of pathologies such as amyotrophic lateral 80 sclerosis [28], epilepsy [13] and Alzheimer's disease [29]. The main targets mostly include 81 subcortical structures, including thalamus [30,31], hippocampus [32,33], nucleus accumbens [34], 82 and pedunculopontine nucleus [35], among others [10]. In addition to the practical advantage of 83 dealing with well-defined structures, the focus on the subcortex is due mainly to its implications 84 in neurological diseases and psychiatric disorders.…”

“…estimating MRI contrast from the histology (or vice versa) to reduce the registration to 517 an easier intra-modality problem. Recent advances with architectures based on generative 518 adversarial networks have shown great potential for this specific problem [80,81], even with 519 specific applications to medical imaging [82]. In this paper we have presented, as a preliminary practical application of our pipeline, the chance 522 of exploring histological sections through the related MRI volume.…”