On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles

Zhang, Qingzhao; Hu, Shengtuo; Sun, Jiachen; Chen, Qi Alfred; Mao, Z. Morley

doi:10.48550/arxiv.2201.05057

Cited by 5 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the safetycritical scenarios from [82], [83] designs a comprehensive open-source toolbox to train and evaluate RL motion planners for autonomous vehicles with customized configuration from users. To obtain a robust trajectory prediction model, [78] generate adversarial trajectory by perturbing existing trajectory with feasible constraints.…”

Section: Constraint Optimizationmentioning

confidence: 99%

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding¹,

Xu²,

Arief³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous driving systems have witnessed a significant development during the past years thanks to the advance in machine learning-enabled sensing and decision-making algorithms. One critical challenge for their massive deployment in the real world is their safety evaluation. Most existing driving systems are still trained and evaluated on naturalistic scenarios collected from daily life or heuristically-generated adversarial ones. However, the large population of cars, in general, leads to an extremely low collision rate, indicating that the safety-critical scenarios are rare in the collected real-world data. Thus, methods to artificially generate scenarios become crucial to measure the risk and reduce the cost. In this survey, we focus on the algorithms of safety-critical scenario generation in autonomous driving. We first provide a comprehensive taxonomy of existing algorithms by dividing them into three categories: data-driven generation, adversarial generation, and knowledge-based generation. Then, we discuss useful tools for scenario generation, including simulation platforms and packages. Finally, we extend our discussion to five main challenges of current works -fidelity, efficiency, diversity, transferability, controllability -and research opportunities lighted up by these challenges.

show abstract

Section: Constraint Optimizationmentioning

confidence: 99%

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding¹,

Xu²,

Arief³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent research [16] demonstrates that trajectory prediction in autonomous driving can be fooled by the adversarial behavior of a surrounding vehicle in both white-box and black-box settings, where adversarial behavior is optimized with Projected Gradient Decent(PGD) [2] or Particle Swarm Optimization(PSO) [17], respectively. Hard constraints are applied to the maximum deviation of way points so as to make the adversarial trajectory physically feasible and not perform unrealistic In this work, we consider such attacks and assume the worst setting, i.e., the attacker has full knowledge of the target system and tries to maximize attack impact.…”

Section: Adversarial Attacks On Trajectory Predictionmentioning

confidence: 99%

“…2, three types of adversarial attacks are presented and we observe that they can cause significant and directional errors. [16] proposes directional error metrics for the optimization of targeted attacks, as shown in Eq. ( 1)…”

Section: Adversarial Attacks On Trajectory Predictionmentioning

confidence: 99%

“…Attack Settings. In the experiments, we study three different types of attacks [16] to the vehicle trajectory prediction algorithms -they are lateral directional attack (shift to the right), longitudinal directional attack (shift forward or backward), and the ADE attack (deviate randomly). The models are adversarially trained on the three types of attacks, respectively.…”

Section: Experiments Setupmentioning

confidence: 99%

“…Some works [11,12,13,14] utilize advanced feature extractors such as graph neural networks or transformers, and the work in [15] proposes a behavior-aware trajectory generator. However, few works studied the robustness of vehicle trajectory prediction, which is in fact of vital importance because 1) autonomous driving is a safety-critical task by nature, 2) recent work [16] demonstrates that the prediction module is vulnerable to adversarial attacks if surrounding vehicles drive along a seemingly natural but crafted trajectory, and 3) current prediction models are often overfitted on the limited patterns in the datasets but suffer the long-tailed distribution of driving scenarios and behaviors. As shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Semi-supervised Semantics-guided Adversarial Training for Trajectory Prediction

Jiao¹,

Sato²,

Chen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Predicting the trajectories of surrounding objects is a critical task in self-driving vehicles and many other autonomous systems. Recent works demonstrate that adversarial attacks on trajectory prediction, where small crafted perturbations are introduced to history trajectories, may significantly mislead the prediction of future trajectories and ultimately induce unsafe planning. However, few works have addressed enhancing the robustness of this important safety-critical task. In this paper, we present the first adversarial training method for trajectory prediction.Compared with typical adversarial training on image tasks, our work is challenged by more random input with rich context and a lack of class labels. To address these challenges, we propose a method based on a semi-supervised adversarial autoencoder, which models disentangled semantic features with domain knowledge and provides additional latent labels for the adversarial training. Extensive experiments with different types of attacks demonstrate that our semi-supervised semantics-guided adversarial training method can effectively mitigate the impact of adversarial attacks and generally improve the system's adversarial robustness to a variety of attacks, including unseen ones. We believe that such semantics-guided architecture and advancement on robust generalization is an important step for developing robust prediction models and enabling safe decision making.Preprint. Under review.

show abstract

Vehicle Sideslip Trajectory Prediction Based on Time-Series Analysis and Multi-Physical Model Fusion

Cao,

Luo,

Wang

et al. 2023

J. Int. Con. Veh.

View full text Add to dashboard Cite

On highways, vehicles that swerve out of their lane due to sideslip can pose a serious threat to the safety of autonomous vehicles. To ensure their safety, predicting the sideslip trajectories of such vehicles is crucial. However, the scarcity of data on vehicle sideslip scenarios makes it challenging to apply data-driven methods for prediction. Hence, this study uses a physical model-based approach to predict vehicle sideslip trajectories. Nevertheless, the traditional physical model-based method relies on constant input assumption, making its long-term prediction accuracy poor. To address this challenge, this study presents the time-series analysis and interacting multiple model-based (IMM) sideslip trajectory prediction (TSIMMSTP) method, which encompasses time-series analysis and multi-physical model fusion, for the prediction of vehicle sideslip trajectories. Firstly, we use the proposed adaptive quadratic exponential smoothing method with damping (AQESD) in the time-series analysis module to predict the input state sequence required by kinematic models. Then, we employ an IMM approach to fuse the prediction results of various physical models. The implementation of these two methods allows us to significantly enhance the long-term predictive accuracy and reduce the uncertainty of sideslip trajectories. The proposed method is evaluated through numerical simulations in vehicle sideslip scenarios, and the results clearly demonstrate that it improves the long-term prediction accuracy and reduces the uncertainty compared to other model-based methods.

show abstract

On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles

Cited by 5 publications

References 19 publications

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Semi-supervised Semantics-guided Adversarial Training for Trajectory Prediction

Vehicle Sideslip Trajectory Prediction Based on Time-Series Analysis and Multi-Physical Model Fusion

Contact Info

Product

Resources

About