Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models

Abstract

Semantic image segmentation is a core task for autonomous driving, which is performed by deep models. Since training these models draws to a curse of human-based image labeling, the use of synthetic images with automatically generated labels together with unlabeled real-world images is a promising alternative. This implies addressing an unsupervised domain adaptation (UDA) problem. In this paper, we propose a new co-training procedure for synth-to-real UDA of semantic segmentation models. It performs iterations where the (unlabeled) real-world training images are labeled by intermediate deep models trained with both the (labeled) synthetic images and the real-world ones labeled in previous iterations. More specifically, a self-training stage provides two domain-adapted models and a model collaboration loop allows the mutual improvement of these two models. The final semantic segmentation labels (pseudo-labels) for the real-world images are provided by these two models. The overall procedure treats the deep models as black boxes and drives their collaboration at the level of pseudo-labeled target images, i.e., neither modifying loss functions is required, nor explicit feature alignment. We test our proposal on standard synthetic and real-world datasets for onboard semantic segmentation. Our procedure shows improvements ranging from approximately 13 to 31 mIoU points over baselines.

Understanding the Influence of Rendering Parameters in Synthetic Datasets for Neural Semantic Segmentation Tasks

“…While previous research has been focused on what impact the synthetic image quality has in object detection tasks, a notable gap still remains in understanding how synthetic data could affect neural network performance in semantic segmentation for autonomous driving (G ómez et al, 2023;Khan et al, 2019).…”

Explore Innovative Depth Vision Models with Domain Adaptation

“…The latest advances in Unsupervised Domain Adaptation (UDA) in image processing have been attempted and progressed in various fields. Goel et al [9] achieved unsupervised domain adaptation by guiding transfer learning and employing the Jensen-Shannon (JS) divergence method. In the remote sensing domain, Elshamli et al [10] introduced an innovative approach to domain adaptation, incorporating denoising autoencoders and domain adversarial neural networks, especially in the classification of hyperspectral and multispectral images.…”