CoSMix: Compositional Semantic Mix for Domain Adaptation in 3D LiDAR Segmentation

Abstract

Deep-learning models for 3D point cloud semantic segmentation exhibit limited generalization capabilities when trained and tested on data captured with different sensors or in varying environments due to domain shift. Domain adaptation methods can be employed to mitigate this domain shift, for instance, by simulating sensor noise, developing domain-agnostic generators, or training point cloud completion networks. Often, these methods are tailored for range view maps or necessitate multi-modal input. In contrast, domain adaptation in the image domain can be executed through sample mixing, which emphasizes input data manipulation rather than employing distinct adaptation modules. In this study, we introduce compositional semantic mixing for point cloud domain adaptation, representing the first unsupervised domain adaptation technique for point cloud segmentation based on semantic and geometric sample mixing. We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world). Each branch operates within one domain by integrating selected data fragments from the other domain and utilizing semantic information derived from source labels and target (pseudo) labels. Additionally, our method can leverage a limited number of human point-level annotations (semi-supervised) to further enhance performance. We assess our approach in both synthetic-to-real and real-to-real scenarios using LiDAR datasets and demonstrate that it significantly outperforms state-of-the-art methods in both unsupervised and semi-supervised settings.

Single Domain Generalization for LiDAR Semantic Segmentation

“…1) Experiments in the DGLSS Setting: We compare our proposed method with a domain adaptation method COS-MIX [7], and domain generalization methods IBN-Net [38], MLDG [39] and DGLSS [10] in the domain generalization experimental setting as introduced in DGLSS [10]. We utilize the Waymo [4], SemanticKITTI [5], and nuScenes [6] datasets.…”

“…Constructing training datasets for every individual sensor and location to mitigate this degradation in performance is prohibitively expensive. In the quest for cost-effective performance maintenance, the spotlight has shifted towards unsupervised domain adaptation [7], [9], [25]- [27]. This strategy aims to leverage both the labeled source datasets and unlabeled target datasets to maintain the performance in the target domain.…”

“…number of beams and Field of View (FoV). Numerous unsupervised domain adaptation studies [7]- [9] offer solutions but demand separate fine-tuning before deployment. In contrast, models trained with robust domain generalization methods hold greater promise for real-time autonomous driving systems.…”

Teachers in Concordance for Pseudo-Labeling of 3D Sequential Data

“…Unsupervised Domain Adaptation (UDA) for Point Cloud Segmentation. UDA for point cloud segmentation can be classified into two setups: (1) simulation-to-real [21], [33] and (2) real-to-real [31], [28], [20]. Simulation-to-real UDA is used when a deep learning-based model is trained with source domain from simulated or synthetically generated data and then tested on a target domain real-world data.…”

Exploring metadiscourse in university lectures in Hong Kong

“…A holistic perception of 3D scenes is crucial for safe autonomous driving [4,8,35,53,63]. Various LiDAR segmentation models have been proposed, with distinct focuses on aspects include LiDAR representations [21, 76, 93-95, 105, 123, 130], model architectures [1,18,25,37,48,54,82,114], sensor fusion [19,64,66,115,131], post-processing [113,125], data aug-mentations [77,86,107], etc. Most recently, researchers started to explore data efficiency [51,58], annotation efficiency [59,65,67,88,99], annotation-free learning [11,12,124], zero-shot learning [13,71], domain adaptation [7,41,50,56,75,81,108], and robustness [49] in LiDAR segmentation, shedding lights for practitioners.…”