Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluation

Abstract

Existing neural network-based autonomous systems are shown to be vulnerable against adversarial attacks, therefore sophisticated evaluation on their robustness is of great importance. However, evaluating the robustness only under the worst-case scenarios based on known attacks is not comprehensive, not to mention that some of them even rarely occur in the real world. In addition, the distribution of safety-critical data is usually multimodal, while most traditional attacks and evaluation methods focus on a single modality. To solve the above challenges, we propose a flow-based multimodal safety-critical scenario generator for evaluating decision-making algorithms. The proposed generative model is optimized with weighted likelihood maximization and a gradient-based sampling procedure is integrated to improve the sampling efficiency. The safety-critical scenarios are generated by querying the task algorithms and the log-likelihood of the generated scenarios is with respect to the risk. Experiments on a self-driving task demonstrate our advantages in terms of testing efficiency and multimodal modeling capability. We evaluate six Reinforcement Learning algorithms with our generated traffic scenarios and provide empirical conclusions about their robustness.

Diverse Critical Interaction Generation for Planning and Planner Evaluation

“…Creating critical test scenarios for testing is one popular direction [2]. [1], [2] use adaptive sampling to generate multi-modal safe-critical initial conditions in cyclists-vehicle interactions. [18] generates critical scenarios with evolutionary algorithms.…”

“…In the planning setting, as discussed before, the route generator can be used to plan the trajectory of vehicle V 1 given past observations and the estimated goal of an unknown driver V 2 . The first dimension of V 1 's style variable q (1)…”

“…q takes value from [−2, 2] c , where c is the dimension of q. As we have discussed before, we use the first dimension of q, denoted as q (1) , to represent the critical rate of the interaction. z is a random vector and is sampled from normal distribution.…”

“…By varying the environment and adversary agents in test cases, we can figure out possible failure modes and improve the decision-making algorithms. A common approach in previous test case generation methods is to sample diverse initial states of the adversary agents [1], [2]. However, one drawback of this approach is that they usually assume over-simplified adversary agents with little reaction to the tested vehicle.…”

ARC: Adversarially Robust Control Policies for Autonomous Vehicles

“…Concurrently, several approaches have emerged in autonomous vehicle testing, where an adversarial policy is used to control an agent (e.g. vehicle, pedestrian) on the road, and aim to find behaviours which cause the target autonomous vehicle to make mistakes [10], [24]- [26]. This type of adversarial testing has been shown to be effective in the validation of autonomous vehicle control policies, by finding weaknesses which may not have been found through traditional validation methods [27], [28].…”