Unsupervised Domain Adaptation in LiDAR Semantic Segmentation with Self-Supervision and Gated Adapters

Abstract

In this paper, we focus on a less explored, but more realistic and complex problem of domain adaptation in LiDAR semantic segmentation. There is a significant drop in performance of an existing segmentation model when training (source domain) and testing (target domain) data originate from different LiDAR sensors. To overcome this shortcoming, we propose an unsupervised domain adaptation framework that leverages unlabeled target domain data for self-supervision, coupled with an unpaired mask transfer strategy to mitigate the impact of domain shifts. Furthermore, we introduce gated adapter modules with a small number of parameters into the network to account for target domain-specific information. Experiments adapting from both real-to-real and synthetic-to-real LiDAR semantic segmentation benchmarks demonstrate the significant improvement over prior arts.

Enhanced Prototypical Learning for Unsupervised Domain Adaptation in LiDAR Semantic Segmentation

“…Recently, Complete & Label [2] proposed a new benchmark on LiDAR UDA and a completion based method to deal with the different sampling patterns of different sensors. [16] came up with a range image-based method that tackles part of the scenario proposed by [2] with the help of self-supervised auxiliary tasks. Our method outperforms [2], [16] in performance, while using a simple method and operating at a runtime that is suited for real-time applications.…”

“…[16] came up with a range image-based method that tackles part of the scenario proposed by [2] with the help of self-supervised auxiliary tasks. Our method outperforms [2], [16] in performance, while using a simple method and operating at a runtime that is suited for real-time applications.…”

Domain Adaptation in LiDAR Semantic Segmentation via Alternating Skip Connections and Hybrid Learning

“…The inputs to HYLDA are two (source and target) LiDAR 3D point clouds and available labels. The point clouds are converted and normalized into spherical projection range view (RV) images of size 64 × 2048 × 5 (X, Y, Z, range, remission) as done in [25], [23], [26], [27].…”

“…We initialize f target with the weights from the pre-trained f Ref Src , and split it into encoder f enc and decoder so we can train f enc , that is coupled with an auxiliary decoder Dec aux (see Fig. 1) using self-supervision [27], [30]. Dec aux is a SALSANext decoder with a tanh activation at the output, which is adapted to work on a simple auxiliary identity reconstruction task.…”

“…Since these datasets are organized differently, a class mapping step is usually needed to compare domain adaptation performance on these datasets. We follow prior work [27] for this and find 11 overlapping object classes in these datasets. The object classes are: { Car, Bicycle, Motorcycle, Other vehicle, Pedestrian, Truck, Drivable surface, Sidewalk, Terrain, Vegetation, Man-made }.…”