Bidirectional Learning for Domain Adaptation of Semantic Segmentation

Abstract

Domain adaptation for semantic image segmentation is very necessary since manually labeling large datasets with pixel-level labels is expensive and time consuming. Existing domain adaptation techniques either work on limited datasets, or yield not so good performance compared with supervised learning. In this paper, we propose a novel bidirectional learning framework for domain adaptation of segmentation. Using the bidirectional learning, the image translation model and the segmentation adaptation model can be learned alternatively and promote to each other. Furthermore, we propose a self-supervised learning algorithm to learn a better segmentation adaptation model and in return improve the image translation model. Experiments show that our method is superior to the state-of-the-art methods in domain adaptation of segmentation with a big margin. The source code is available at https://github.com/liyunsheng13/BDL.On the forward direction (i.e., "translation-tosegmentation", similar to [12,36]), we propose a self-supervised learning (SSL) approach in training our segmentation adaptation model. Different from segmentation models trained on real data, the segmentation adaptation model is trained on both synthetic and real datasets, but the real data has no annotations. At every time, we may regard the predicted labels for real data with high confidence as the approximation to the ground truth

Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation

“…artificial light sources casting very different illumination patterns at night). A major class of adaptation approaches, including [5,12,13,19,28,31,34,36,43,46], involves adversarial confusion or feature alignment between domains. The general concept of curriculum learning has been applied to domain adaptation by ordering tasks [42] or target-domain pixels [47], while we order domains.…”

“…where we have used the definition of IoU as well as (13) in the second line, (19) in the third line, (20) in the fourth line, and the definition of UIoU in the last line.…”

Improving Land Cover Segmentation Across Satellites Using Domain Adaptation

“…Chang et al [35] proposed a domain-invariant structure extraction (DISE) framework to disentangle images into domain-invariant structure and domain-specific texture representations. Li et al [36] proposed adding an extra step called bidirectional learning (BDL). The principle behind this step is to alternate between segmentation learning and image translation, which is supervised using the segmentation model.…”

Instance Segmentation with Unsupervised Adaptation to Different Domains for Autonomous Vehicles

“…Cycada [32] trains in Synthetic data and then transforms the real images into a synthetic style by using GANs. [33] and [34] also use a similar approach for semantically segmenting the scenes. It should be noted that this requires additional computation to pre-process the images.…”