A simulation-based framework for functional testing of automated driving controllers

Abstract

Motion planning is a major component of any automated driving system. The safety assessment of such components requires a formal characterization of the perception and control mechanisms. This requires dedicated tools and models for the environment, sensors and vehicles that are highly representative of the real world. Simulation is a method to virtually investigate the behavior of systems under study. It has a key role to play in demonstrating the safety of autonomous vehicles. In this context, we consider a control module as a black-box and try to determine a reference which represents the 'right decision', if it exists. An optimization-based reference model is created for the control function. This model allows each scene in the environment to be mapped to the desired decision regardless of the black-box. The black-box and the reference model are run on several critical scenarios. In output, an assessment of decision making is performed along with systematic criticality characterization of targeted scenarios.

A Survey on Automated Driving System Testing: Landscapes and Trends

“…Optimization-based oracle model The model can generate an oracle area in a given scenario [104] relative speed of the two vehicles in the collision is minimal. The obtained scenarios can be taken as the behavioral boundary that divides the safe and unsafe behaviors.…”

“…Like the planning module, the control module also faces the oracle problem in its testing-indeed, it is usually not straightforward to determine whether a control decision is "correct". In [104], Djoudi et al propose a framework to determine whether the control module makes "the correct decision". They design a model to generate an oracle area in the given scenario, which is the closest safe position ahead of the vehicle.…”

“…The collected studies adopt two major techniques for testing the control module, including fault injection [101] and falsification [102,103]. To tackle the oracle problem in control module testing, the framework proposed in [104] could generate an oracle area for judging whether the control decision is "correct", and the corresponding threat is denoted as implementation capability in Fig. 2.…”

“…LGSVL [213] High-fidelity simulator supporting sensor simulation, vehicle dynamics and control simulation, etc. [21,92,125,138,142,145,[179][180][181][182]225] 2020 SCANeR Studio [248] Flexible simulator allowing simulation configuration and results analysis [104] 2020 ADAMS [215] Supporting the analysis of vehicle dynamics -2020…”