Taking a Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation

Abstract

We consider the problem of unsupervised domain adaptation in semantic segmentation. A key in this campaign consists in reducing the domain shift, i.e., enforcing the data distributions of the two domains to be similar. One of the common strategies is to align the marginal distribution in the feature space through adversarial learning. However, this global alignment strategy does not consider the category-level joint distribution. A possible consequence of such global movement is that some categories which are originally well aligned between the source and target may be incorrectly mapped, thus leading to worse segmentation results in target domain. To address this problem, we introduce a category-level adversarial network, aiming to enforce local semantic consistency during the trend of global alignment. Our idea is to take a close look at the category-level joint distribution and align each class with an adaptive adversarial loss. Specifically, we reduce the weight of the adversarial loss for category-level aligned features while increasing the adversarial force for those poorly aligned. In this process, we decide how well a feature is category-level aligned between source and target by a co-training approach. In two domain adaptation tasks, i.e., GTA5 → Cityscapes and SYN-THIA → Cityscapes, we validate that the proposed method matches the state of the art in segmentation accuracy.

Constructing Self-Motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach

“…We compare the proposed Py-CDA with several state-of-the-art methods. A majority of them uses adversarial training to bring closer the source and target domains on the feature level (ROAD [4]), on both features and pixels (FCAN [40], CyCADA [17]), on the output maps (OutputAdapt [35], CLAN [26]), and combining adversarial learning with entropy minimization (AD-VENT [36]). In contrast, our PyCDA approach, along with CDA [38], and ST [43], adapts the neural networks by posterior regularization instead.…”

“…In this work, we also view the self-training [43] as a curriculum-style domain adaptation method. The other line of work is to reduce the domain shift in feature space or output space and tries to seek a better way to align both domains in an intermediate layer [4,18,40,35,27,42,5,17,19,33,26]. In [18] Figure 2: Overview of our self-motivated pyramid curriculum domain adaptation (PyCDA) approach to segmentation.…”

“…In [4,40], they focus on utilizing spatial information to help the discriminator distinguish the domains better. In [26,42], they focus on solving the problem of class boundary or outliers in the domain adaptation. In [17,27,33], they propose to learn domain adaptive segmentation networks through directly translating the source images to the target ones at the pixel level.…”

“…In order to better take advantage of the synthetic imagery for the semantic segmentation of real scenes, we pro-arXiv:1908.09547v1 [cs.CV] 26 Aug 2019 pose a novel domain adaptation approach by drawing an intriguing connection between two previous works: curriculum domain adaptation [39,38] and self-training [43]. This connection naturally leads to self-motivated pyramid curriculums and a new training algorithm for the cross-domain adaptation of semantic segmentation networks.…”

Domain-Agnostic Learning With Anatomy-Consistent Embedding for Cross-Modality Liver Segmentation

“…However, anatomy-consistency is not always guaranteed without explicitly enforcing semantic consistency on content space. As for feature-level adaptation, while it seems effective for tasks like classification, it is unclear how well it might scale to dense structured domain adaptation [19,16].…”