Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation

Xiao, Aoran; Huang, Jiaxing; Guan, Dayan; Zhan, Fangneng; Lu, Shijian

doi:10.48550/arxiv.2107.05399

Cited by 4 publications

(3 citation statements)

References 49 publications

(83 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following this direction, [27] combines Gaussian Process (GP) and Dirichlet Process (DP) to build a DP-GP model with an infinite number of clusters to discover traffic primitives, which can be combined to create new scenarios. [38] generates point cloud sequential datasets via minimizing the gap between real-world LiDAR and simulation data. Similarly, Meta-sim [39] and Meta-Sim2 [40] try to minimize the sim-toreal gap to reconstruct traffic scenarios for automatic labeling.…”

Section: Bayesian Networkmentioning

confidence: 99%

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding¹,

Xu²,

Arief³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous driving systems have witnessed a significant development during the past years thanks to the advance in machine learning-enabled sensing and decision-making algorithms. One critical challenge for their massive deployment in the real world is their safety evaluation. Most existing driving systems are still trained and evaluated on naturalistic scenarios collected from daily life or heuristically-generated adversarial ones. However, the large population of cars, in general, leads to an extremely low collision rate, indicating that the safety-critical scenarios are rare in the collected real-world data. Thus, methods to artificially generate scenarios become crucial to measure the risk and reduce the cost. In this survey, we focus on the algorithms of safety-critical scenario generation in autonomous driving. We first provide a comprehensive taxonomy of existing algorithms by dividing them into three categories: data-driven generation, adversarial generation, and knowledge-based generation. Then, we discuss useful tools for scenario generation, including simulation platforms and packages. Finally, we extend our discussion to five main challenges of current works -fidelity, efficiency, diversity, transferability, controllability -and research opportunities lighted up by these challenges.

show abstract

Section: Bayesian Networkmentioning

confidence: 99%

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding¹,

Xu²,

Arief³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Lots of works use Conditional Random Fields (CRFs) [33,61,20,34] or random walk [66] to propagate labels. Moreover, there are also works that utilize prototype learning [33,66], siamese learning [61,44], temporal constraints [36], smoothness constraints [44,47], attention [54,66], cross task consistency [44], and synthetic data [56] to help training.…”

Section: Label-efficient 3d Semantic Segmentationmentioning

confidence: 99%

LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds

Liu¹,

Yin²,

Qi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Semantic segmentation of LiDAR point clouds is an important task in autonomous driving. However, training deep models via conventional supervised methods requires large datasets which are costly to label. It is critical to have label-efficient segmentation approaches to scale up the model to new operational domains or to improve performance on rare cases. While most prior works focus on indoor scenes, we are one of the first to propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning and is applicable to nearly any 3D semantic segmentation backbones. Specifically, we leverage geometry patterns in outdoor scenes to have a heuristic pre-segmentation to reduce the manual labeling and jointly design the learning targets with the labeling process. In the learning step, we leverage prototype learning to get more descriptive point embeddings and use multi-scan distillation to exploit richer semantics from temporally aggregated point clouds to boost the performance of single-scan models. Evaluated on the SemanticKITTI and the nuScenes datasets, we show that our proposed method outperforms existing label-efficient methods. With extremely limited human annotations (e.g., 0.1% point labels), our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.

show abstract

“…With transfer learning, a common two-step approach to overcome the scarcity of data is to train a model on large volumes of synthetically generated data and, subsequently, adopt this model by using a small sample of real data [18]. Although this process has been widely applied in computer vision [19], there is limited research on its application to improve the robustness and accuracy of vehicle detection models for low-cost manufacturing LiDAR sensors. In the literature, there are several types of transfer learning, including homogenous, heterogenous, inductive, and transductive transfer learning [20,21], and there are different methods used for transfer learning, namely feature-based and model-based transfer learning [20].…”

Section: Introductionmentioning

confidence: 99%

A Robust Vehicle Detection Model for LiDAR Sensor Using Simulation Data and Transfer Learning Methods

Lakshmanan,

Roach,

Giannetti

et al. 2023

View full text Add to dashboard Cite

Vehicle detection in parking areas provides the spatial and temporal utilisation of parking spaces. Parking observations are typically performed manually, limiting the temporal resolution due to the high labour cost. This paper uses simulated data and transfer learning to build a robust real-world model for vehicle detection and classification from single-beam LiDAR of a roadside parking scenario. The paper presents a synthetically augmented transfer learning approach for LiDAR-based vehicle detection and the implementation of synthetic LiDAR data. A synthetic augmented transfer learning method was used to supplement the small real-world data set and allow the development of data-handling techniques. In addition, adding the synthetically augmented transfer learning method increases the robustness and overall accuracy of the model. Experiments show that the method can be used for fast deployment of the model for vehicle detection using a LIDAR sensor.

show abstract

Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation

Cited by 4 publications

References 49 publications

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds

A Robust Vehicle Detection Model for LiDAR Sensor Using Simulation Data and Transfer Learning Methods

Contact Info

Product

Resources

About