Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation

Abstract

A deep learning model trained on some labeled data from a certain source domain generally performs poorly on data from different target domains due to domain shifts. Unsupervised domain adaptation methods address this problem by alleviating the domain shift between the labeled source data and the unlabeled target data. In this work, we achieve cross-modality domain adaptation, i.e. between CT and MRI images, via disentangled representations. Compared to learning a one-toone mapping as the state-of-art CycleGAN, our model recovers a manyto-many mapping between domains to capture the complex cross-domain relations. It preserves semantic feature-level information by finding a shared content space instead of a direct pixelwise style transfer. Domain adaptation is achieved in two steps. First, images from each domain are embedded into two spaces, a shared domain-invariant content space and a domain-specific style space. Next, the representation in the content space is extracted to perform a task. We validated our method on a crossmodality liver segmentation task, to train a liver segmentation model on CT images that also performs well on MRI. Our method achieved Dice Similarity Coefficient (DSC) of 0.81, outperforming a CycleGAN-based method of 0.72. Moreover, our model achieved good generalization to joint-domain learning, in which unpaired data from different modalities are jointly learned to improve the segmentation performance on each individual modality. Lastly, under a multi-modal target domain with significant diversity, our approach exhibited the potential for diverse image generation and remained effective with DSC of 0.74 on multi-phasic MRI while the CycleGAN-based method performed poorly with a DSC of only 0.52.

Domain-Agnostic Learning With Anatomy-Consistent Embedding for Cross-Modality Liver Segmentation

“…DSC (std) CycleGAN [26] 0.721 (0.049) TD-GAN [25] 0.793 (0.066) DADR [24] 0.806 (0.035) DALACE 0.847 (0.041) and tested on pre-phase MR to serve as the lowerbound for each task. Please see Table 2 for details.…”

“…Settings For each cross-validation split, four folds of CT data with segmentation masks and pre-contrast MR data without segmentation masks are used to train, and one fold of pre-contrast MR data without segmentation masks is used to test. The state-of-the-art models CycleGAN [26], TD-GAN [25], and DADR [24] are trained with the same partition of data for the DA task. DALACE finds a shared space to embed both CT and MR and transfers both modalities into anatomical images while CycleGAN and TD-GAN tries to transfer directly between CT and MR.…”

“…DSC (std) CycleGAN [26] 0.522 (0.064) TD-GAN [25] 0.719 (0.089) DADR [24] 0.742 (0.045) DALACE 0.794 (0.044) details. The upperbound might be lower than the actual upperbound given the noisy and small MR dataset for training with 5-fold cross-validation.…”

“…Instead, these models learn an arbitrary one-to-one mapping and generate translated output lacking in diversity [1,9]. DADR [24] achieves DA by disentangling medical images into content space and style space. However, anatomy-consistency is not always guaranteed without explicitly enforcing semantic consistency on content space.…”

“…Furthermore, the translation in pixel-level information by making the marginal distributions of the two domains as similar as possible does not necessarily guarantee semantic consistency [21]. This is also the case for methods that incorporate feature-level marginal distributions alignment which do not explicitly enforce semantic-consistency, such as DADR [24].…”