CrDoCo: Pixel-Level Domain Transfer With Cross-Domain Consistency

Abstract

Semantic seg. Depth prediction Optical flow Labeled examples (source domain) Input (target domain) Output Figure 1: Applications of the proposed method. Our method has the applications ranging from semantic segmentation (top row), depth prediction (middle row), to optical flow estimation (bottom row). AbstractUnsupervised domain adaptation algorithms aim to transfer the knowledge learned from one domain to another (e.g., synthetic to real images). The adapted representations often do not capture pixel-level domain shifts that are crucial for dense prediction tasks (e.g., semantic segmentation). In this paper, we present a novel pixel-wise adversarial domain adaptation algorithm. By leveraging image-toimage translation methods for data augmentation, our key insight is that while the translated images between domains may differ in styles, their predictions for the task should be consistent. We exploit this property and introduce a crossdomain consistency loss that enforces our adapted model to produce consistent predictions. Through extensive experimental results, we show that our method compares favorably against the state-of-the-art on a wide variety of unsupervised domain adaptation tasks.

Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-Segmentation

“…In image-to-image translation, enforcing cycle consistency allows the model to learn the mappings between domains without paired data [55], [56], [60]. In unsupervised domain adaptation, exploiting cross-domain invariance in the label space results in more consistent task predictions for unlabeled images of different domains [61]. In motion analysis, enforcing forward-backward consistency constraints has been shown effective for detecting occlusion while learning optical flow [58], [62] or enforcing temporal consistency in videos [63].…”

Unsupervised domain adaptation for mobile semantic segmentation based on cycle consistency and feature alignment

“…Unfortunately, the authors were not able to train the full framework with all the proposed modules, due to hardware constraints hindering the insertion of a fully convolutional predictor (i.e., the segmentation network for semantic consistency) inside an already memory-demanding generative module comprising four neural networks. Similarly to [25], Chen et al [30] propose an extension of the CycleGAN framework by introducing a pair of feature domain discriminators and a couple of semantic segmentation networks.…”

Adversarial Learning and Self-Teaching Techniques for Domain Adaptation in Semantic Segmentation

“…In particular the latter is the model employed for the generator network in this work. All the approaches for generic images can be applied also to road scenes, however since this is a very relevant application [11], [12] there has been a large effort both in the acquisition of datasets [13]- [15] and in the development of ad-hoc approaches [16]- [18].…”