Model Validation and Scenario Selection for Virtual-Based Homologation of Automated Vehicles

Abstract

Due to the rapid progress in the development of automated vehicles over the last decade, their market entry is getting closer. One of the remaining challenges is the safety assessment and type approval of automated vehicles, as conventional testing in the real world would involve an unmanageable mileage. Scenario-based testing using simulation is a promising candidate for overcoming this approval trap. Although the research community has recognized the importance of safeguarding in recent years, the quality of simulation models is rarely taken into account. Without investigating the errors and uncertainties of models, virtual statements about vehicle safety are meaningless. This paper describes a whole process combining model validation and safety assessment. It is demonstrated by means of an actual type-approval regulation that deals with the safety assessment of lane-keeping systems. Based on a thorough analysis of the current state-of-the-art, this paper introduces two approaches for selecting test scenarios. While the model validation scenarios are planned from scratch and focus on scenario coverage, the type-approval scenarios are extracted from measurement data based on a data-driven pipeline. The deviations between lane-keeping behavior in the real and virtual world are quantified using a statistical validation metric. They are then modeled using a regression technique and inferred from the validation experiments to the unseen virtual type-approval scenarios. Finally, this paper examines safety-critical lane crossings, taking into account the modeling errors. It demonstrates the potential of the virtual-based safeguarding process using exemplary simulations and real driving tests.

Virtual Testing of Automated Driving Systems. A Survey on Validation Methods

“…Taking into account distribution functions allows, for instance, considering simulation stochasticity with no need of averaging over the repetitions, hence preserving a higher amount of information. Such an approach was exploited in [51] to derive validation metrics for a lane-keeping application.…”

“…Several proposals are also found within the scientific literature, in particular in [39], [45], [51]. In [45], the authors study the fidelity level that a Vehicle-Hardware-in-the-Loop (Ve-HIL) setup can deliver with respect to a MiL environment by comparing the real-world experiments against the evidence generated by simulation environments.…”

“…Overall, the VeHIL returned significantly better fidelity than the MiL. In [51], a method is suggested which firstly checks validation scenario coverage and then computes the correlation between the simulated and logged lateral acceleration for a lanekeeping application by also considering stochastic effects of the particular VeHIL environment. Similarly to [45], in [39] a VeHIL setup is investigated of the achievable fidelity level for increasing complexity driving scenarios using camera stimulation rather than signal injection.…”

A Credibility Assessment Approach for Scenario-Based Virtual Testing of Automated Driving Functions

“…In [37] the authors propose an approach for appropriate scenario selection and model validation for the virtual testing of ADFs. At first they present a coverage-based and datadriven approach for scenario generation for their simulation pipeline.…”

The Future of Autonomous Vehicles in the Opinion of Automotive Market Users

“…The technology of the fully autonomous vehicle (AV) is best adapted to these megatrends, especially in the case of connected and autonomous vehicles (CAVs), which enable communication between vehicles, infrastructure and other road users (so-called V2X connectivity) [1,2]. Automated driving is a major trend in the automotive industry as it promises to increase road safety and driver comfort [3].…”