AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles

Wang, Jingkang; Pun, Ava; Tu, James; Manivasagam, Sivabalan; Sadat, Abbas; Casas, Sergio; Ren, Mengye; Urtasun, Raquel

doi:10.1109/cvpr46437.2021.00978

Cited by 74 publications

(38 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It assumes the existence of a malicious attacker trying to fail the ego car by tampering with either the environment or the ego car's internal states directly. Regarding the former, the attacker creates adversarial examples or sends malicious signals to fool the ego car's sensor processing models, e.g., perturbing front camera images [16], [17], [18], road signs [19], [20], rendering malicious shapes on the road [21] or billboard [22], spoofing GPS signals [23], spoofing LiDAR signals [24], [25], or influencing both LiDAR and camera inputs [26]. Regarding the latter, an attacker can directly inject faults inside the system to fail it [27], [28], [29], [30], [31], [32], [33], [34], [35].…”

Section: Other Safety Assessment Methodsmentioning

confidence: 99%

A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation

Zhong¹,

Tang²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automated Driving Systems (ADSS) have seen rapid progress in recent years. To ensure the safety and reliability of these systems, extensive testings are being conducted before their future mass deployment. Testing the system on the road is the closest to real-world and desirable approach, but it is incredibly costly. Also, it is infeasible to cover rare corner cases using such real-world testing. Thus, a popular alternative is to evaluate an ADS's performance in some well-designed challenging scenarios, a.k.a. scenario-based testing. High-fidelity simulators have been widely used in this setting to maximize flexibility and convenience in testing what-if scenarios. Although many works have been proposed offering diverse frameworks/methods for testing specific systems, the comparisons and connections among these works are still missing. To bridge this gap, in this work, we provide a generic formulation of scenario-based testing in high-fidelity simulation and conduct a literature review on the existing works. We further compare them and present the open challenges as well as potential future research directions.

show abstract

Section: Other Safety Assessment Methodsmentioning

confidence: 99%

A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation

Zhong¹,

Tang²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Parallel work [51] generates a set of possible driving paths and identifies all the possible safe driving trajectories that can be taken starting at different times. Similarly, [59] directly optimize existing trajectories to perturb the driving paths of surrounding vehicles. They use Bayesian Optimization [111] for the optimization, and the scenario is represented with point cloud.…”

Section: Initial Conditionmentioning

confidence: 99%

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding¹,

Xu²,

Arief³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous driving systems have witnessed a significant development during the past years thanks to the advance in machine learning-enabled sensing and decision-making algorithms. One critical challenge for their massive deployment in the real world is their safety evaluation. Most existing driving systems are still trained and evaluated on naturalistic scenarios collected from daily life or heuristically-generated adversarial ones. However, the large population of cars, in general, leads to an extremely low collision rate, indicating that the safety-critical scenarios are rare in the collected real-world data. Thus, methods to artificially generate scenarios become crucial to measure the risk and reduce the cost. In this survey, we focus on the algorithms of safety-critical scenario generation in autonomous driving. We first provide a comprehensive taxonomy of existing algorithms by dividing them into three categories: data-driven generation, adversarial generation, and knowledge-based generation. Then, we discuss useful tools for scenario generation, including simulation platforms and packages. Finally, we extend our discussion to five main challenges of current works -fidelity, efficiency, diversity, transferability, controllability -and research opportunities lighted up by these challenges.

show abstract

“…We believe that we are the first to take causal interventions in static scenes to test AV detection systems, although multiple approaches (Ghodsi et al, 2021;Abeysirigoonawardena et al, 2019;Koren et al, 2018;Corso et al, 2019;O'Kelly et al, 2018;Rempe et al, 2021) test AV systems through adversarial manipulation of actor trajectories and operate on the planning subsystem. Wang et al (2021a) generates adversarial scenarios for AV systems by black-box optimization of actor trajectory perturbations, simulating LiDAR sensors in perturbed real scenes. Prior research has focused on optimization techniques for adversarial scenario generation through the manipulation of trajectories of vehicles and pedestrians.…”

Section: Related Workmentioning

confidence: 99%

“…The status quo approach to finding these groups in the AV stack operates in hindsight by analyzing real-world scenes requiring driver intervention or by feeding replayed or simulated scenes to a model and finding those that result in poor performance. Advanced techniques may use adversarial attacks to actively find failures (Xie et al, 2017;Athalye et al, 2017;Wang et al, 2021a;Rempe et al, 2021). In all cases, the found data is fed back into the retraining process.…”

Section: Introductionmentioning

confidence: 99%

Causal BERT: Improving object detection by searching for challenging groups

Resnick

Litany

Kar

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

View full text Add to dashboard Cite

Modern computer vision applications rely on learning-based perception modules parameterized with neural networks for tasks like object detection. These modules frequently have low expected error overall but high error on atypical groups of data due to biases inherent in the training process. In building autonomous vehicles (AV), this problem is an especially important challenge because their perception modules are crucial to the overall system performance. After identifying failures in AV, a human team will comb through the associated data to group perception failures that share common causes. More data from these groups is then collected and annotated before retraining the model to fix the issue. In other words, error groups are found and addressed in hindsight. Our main contribution is a pseudo-automatic method to discover such groups in foresight by performing causal interventions on simulated scenes. To keep our interventions on the data manifold, we utilize masked language models. We verify that the prioritized groups found via intervention are challenging for the object detector and show that retraining with data collected from these groups helps inordinately compared to adding more IID data. We also plan to release software to run interventions in simulated scenes, which we hope will benefit the causality community.

show abstract

AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles

Cited by 74 publications

References 21 publications

A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation

A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Causal BERT: Improving object detection by searching for challenging groups

Contact Info

Product

Resources

About