Finding Diverse Failure Scenarios in Autonomous Systems Using Adaptive Stress Testing

Abstract

<div>Identifying and eliminating failure scenarios is critical in the development of autonomous vehicle (AV) systems. However, finding such failures through real-world vehicle-level testing is a difficult task as system disengagements and accidents are rare occurrences. Simulation approaches have been proposed to supplement vehicle-level testing and reduce the costs associated with operating large fleets of autonomous test vehicles. While one can run more vehicles in simulation than in the real world, applying traditional Monte Carlo sampling techniques to find failures still yields an unguided search and a large waste of computing resources. A more directed method than random sampling is needed to identify failure scenarios in a computationally efficient manner. Adaptive Stress Testing (AST) is a method that uses reinforcement learning (RL) paradigms to efficiently find failure scenarios in stochastic sequential decision-making systems. Through iteratively exploring the action space and collecting rewards, AST aims to establish an optimal policy that generates a set of high-probability failure trajectories. However, the trajectories obtained through AST tend to lack diversity and converge to similar failure states. Due to the range of possible accident scenarios an AV can face, such homogeneous failures are not very beneficial in validating a system’s roadworthiness. In this article, we present a method to enhance the expressiveness of the failure scenarios found using AST. By augmenting the reward function used by AST with domain relevant information, we guide the solver to discover more diverse sets of trajectories. We present an implementation using Monte Carlo Tree Search (MCTS). To show the efficacy of our approach, we evaluate the failure trajectories obtained for a vehicle and pedestrian crosswalk scenario. We show that our implementation is able to find more diverse and domain-relevant failures when compared with baseline AST.</div>

Finding Critical Scenarios for Automated Driving Systems: A Systematic Mapping Study

“…In [58], the authors proposed to extend AST by using deep reinforcement learning to solve the MDP and apply it to a set of autonomous vehicle scenarios. The studies in [59] and [96] improve the reward function previously proposed by using RSS (Responsibility Sensitive Safety) [7] to find scenarios where the ego vehicle performs improper actions and including a trajectory dis-similarity reward to find diverse failures. In [60], the authors extend the work performed in [58] by replacing the original Multi-layer Perceptron Network with a Recurrent Neural Network.…”

“…Methods to explore a logical scenario including such parameter trajectories are discussed in this section. Such problems are also called stress testing in [58], [60], [96] or adversarial testing in [105].…”

Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review

“…Methods to explore a logical scenario including such parameter trajectories are discussed in this section. Such problems are also called stress testing in [58], [60], [93] or adversarial testing in [102].…”

“…In [58], the authors proposed to extend AST by using deep reinforcement learning to solve the MDP and apply it to a set of autonomous vehicle scenarios. [59] and [93] improve the reward function previously proposed by using RSS [7] to find scenarios where the ego vehicle performs improper actions and including a trajectory dissimilarity reward to find diverse failures. In [60], the authors extend the work performed in [58] by replacing the original Multi-layer Perceptron Network with a Recurrent Neural Network.…”

A Real-time Critical-scenario-generation Framework for Testing Autonomous Driving System

“…Exploring logical scenarios with parameter trajectories [2], which are also called stress testing in [8], [9], [16] or adversarial testing in [10], is different from the scenario reconstructed with real data [19]- [21]. Exploring logical scenarios with parameter trajectories aims at looking for the failure or likelyfailure situations of an autonomous system, giving boundary situation or proper suggestions of Operational Design Domain for system designers.…”