SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Driving

Yang, Zhan-Ying; Chai, Yuning; Anguelov, Dragomir; Zhou, Yin; Sun, Pei; Erhan, Dumitru; Rafferty, Sean M.; Kretzschmar, Henrik

doi:10.1109/cvpr42600.2020.01113

Cited by 74 publications

(27 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we can easily generate highdimensional sensing scenarios. SurfelGAN is proposed in [34] to directly generate point cloud data to represent scenarios from the view of the AV. [35] can add new vehicles to collected driving videos to generate realistic video scenarios, where they also consider the motion planning of vehicles.…”

Section: Deep Generative Modelsmentioning

confidence: 99%

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding¹,

Xu²,

Arief³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous driving systems have witnessed a significant development during the past years thanks to the advance in machine learning-enabled sensing and decision-making algorithms. One critical challenge for their massive deployment in the real world is their safety evaluation. Most existing driving systems are still trained and evaluated on naturalistic scenarios collected from daily life or heuristically-generated adversarial ones. However, the large population of cars, in general, leads to an extremely low collision rate, indicating that the safety-critical scenarios are rare in the collected real-world data. Thus, methods to artificially generate scenarios become crucial to measure the risk and reduce the cost. In this survey, we focus on the algorithms of safety-critical scenario generation in autonomous driving. We first provide a comprehensive taxonomy of existing algorithms by dividing them into three categories: data-driven generation, adversarial generation, and knowledge-based generation. Then, we discuss useful tools for scenario generation, including simulation platforms and packages. Finally, we extend our discussion to five main challenges of current works -fidelity, efficiency, diversity, transferability, controllability -and research opportunities lighted up by these challenges.

show abstract

Section: Deep Generative Modelsmentioning

confidence: 99%

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding¹,

Xu²,

Arief³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In this case, building autonomous vehicle simulators with high fidelity could help (e.g. [144], [145]). It is an important and challenging future research direction to build such a simulator that could take the predictive uncertainty from probabilistic object detectors as inputs, and evaluate probabilistic detection in a full software stack with novel metrics.…”

Section: Better Evaluation Of Probabilistic Object Detectionmentioning

confidence: 99%

A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving

Feng,

Harakeh,

Waslander

et al. 2020

Preprint

View full text Add to dashboard Cite

Capturing uncertainty in object detection is indispensable for safe autonomous driving. In recent years, deep learning has become the de-facto approach for object detection, and many probabilistic object detectors have been proposed. However, there is no summary on uncertainty estimation in deep object detection, and existing methods are not only built with different network architectures and uncertainty estimation methods, but also evaluated on different datasets with a wide range of evaluation metrics. As a result, a comparison among methods remains challenging, as does the selection of a model that best suits a particular application. This paper aims to alleviate this problem by providing a review and comparative study on existing probabilistic object detection methods for autonomous driving applications. First, we provide an overview of generic uncertainty estimation in deep learning, and then systematically survey existing methods and evaluation metrics for probabilistic object detection. Next, we present a strict comparative study for probabilistic object detection based on an image detector and three public autonomous driving datasets. Finally, we present a discussion of the remaining challenges and future works. Code has been made available at https://github.com/asharakeh/pod_ compare.git.

show abstract

“…Similar in flavour, therefore, to recent approaches proposed for unaligned domain transfer in vision [15], [16], [35], [36], we too consider learning unaligned mappings between simulated world layouts and real radar observations. We model the forward and backward model side-by-side using adversarial and cyclical consistency losses.…”

Section: Related Workmentioning

confidence: 99%

“…We model the forward and backward model side-by-side using adversarial and cyclical consistency losses. Unlike in [15], [16], [35], [36] which consider a deterministic oneto-one mapping between domains, we adopt an inherently probabilistic approach. We capture a distribution over possible power returns to account for the stochastic noise processes arising throughout the radar sensing pipeline.…”

Section: Related Workmentioning

confidence: 99%

There and Back Again: Learning to Simulate Radar Data for Real-World Applications

Weston¹,

Jones²,

Posner³

2020

Preprint

View full text Add to dashboard Cite

Simulating realistic radar data has the potential to significantly accelerate the development of data-driven approaches to radar processing. However, it is fraught with difficulty due to the notoriously complex image formation process. Here we propose to learn a radar sensor model capable of synthesising faithful radar observations based on simulated elevation maps. In particular, we adopt an adversarial approach to learning a forward sensor model from unaligned radar examples. In addition, modelling the backward model encourages the output to remain aligned to the world state through a cyclical consistency criterion. The backward model is further constrained to predict elevation maps from real radar data that are grounded by partial measurements obtained from corresponding lidar scans. Both models are trained in a joint optimisation. We demonstrate the efficacy of our approach by evaluating a down-stream segmentation model trained purely on simulated data in a real-world deployment. This achieves performance within four percentage points of the same model trained entirely on real data.

show abstract

SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Driving

Cited by 74 publications

References 18 publications

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving

There and Back Again: Learning to Simulate Radar Data for Real-World Applications

Contact Info

Product

Resources

About