Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation

Abstract

Self-training is a competitive approach in domain adaptive segmentation, which trains the network with the pseudo labels on the target domain. However inevitably, the pseudo labels are noisy and the target features are dispersed due to the discrepancy between source and target domains. In this paper, we rely on representative prototypes, the feature centroids of classes, to address the two issues for unsupervised domain adaptation. In particular, we take one step further and exploit the feature distances from prototypes that provide richer information than mere prototypes. Specifically, we use it to estimate the likelihood of pseudo labels to facilitate online correction in the course of training. Meanwhile, we align the prototypical assignments based on relative feature distances for two different views of the same target, producing a more compact target feature space. Moreover, we find that distilling the already learned knowledge to a self-supervised pretrained model further boosts the performance. Our method shows tremendous performance advantage over state-of-the-art methods. We will make the code publicly available.

Multiple Fusion Adaptation: A Strong Framework for Unsupervised Semantic Segmentation Adaptation

“…We use co-learning and integrate multiple fusion strategies to resist the noisy pseudo labels, as well as to retain informative samples. Experimental results show that the proposed MFA marginally surpasses [21,22].…”

“…In these works, the labels are all available, which is different from the unsupervised domain adaptation problem. We note that [21,22] also consider the noise issue of pseudo labels on semantic segmentation adaptation. [22] estimates the uncertainty of predictions and reduces the impact of low-confidence samples during pseudo label learning.…”

“…[22] estimates the uncertainty of predictions and reduces the impact of low-confidence samples during pseudo label learning. [21] take this issue by exploiting the feature distances from prototypes. Our method tackles the noisy pseudo label problem from a different viewpoint.…”

“…We achieve the best scores on most categories (11 in 19). Second, under the fair comparison, MFA* presents 58.2% mIoU, with additional model distillation stage [21]. In line with [21], we use the SimCLRv2 [2] pretrained weights and same distillation strategy.…”

“…A popular pipeline adopted by many state-of-the-art methods [19,21,22] consists of two training stages, i.e., a warm-up supervised training on the source domain and a sequential self-training on the target domain. Specifically, the first stage trains a warm-up model on the source domain data.…”