A Credibility Assessment Approach for Scenario-Based Virtual Testing of Automated Driving Functions

Abstract

An immense test space is pushing the development and testing of automated driving functions from real to virtual environments. The virtual world is provided by interconnected simulation models representing sensors, vehicle dynamics, and both static and dynamic environment. For the virtual validation of automated driving, special attention must be paid to the simulation's credibility, which can be impaired by inappropriate or inaccurate simulation models and tools. Therefore, in this work a method is proposed to assess the credibility of simulation-based testing for automated driving. The approach allows a qualitative and relatively quantitative comparisons between scenarios as well as between different simulation setups. Therefore, several uni-and multivariate metrics are applied towards a scoring of similarity of the behavior between simulation and real test drive. This is achieved by using ground truth data in form of simulation scenarios from real world measurement data. In this way, the virtual automated vehicle encounters the same conditions and surroundings than its counterpart in the real world for evaluating their similarity. The practical applicability of the proposed credibility assessment approach is demonstrated in a case study, in which the credibility of an exemplary simulation-based test bench is inferred.

Modeling & Simulation SPICE: Assessing the Capability of Credible Simulation Processes

“…On the other hand, often there is no other way than to answer safety-related questions with the help of simulation, for example in the aerospace or healthcare industry, where the behavior of a space shuttle or a medical device in the respective target environment has to be simulated, because tests under real conditions would practically, economically or ethically not be feasible (prostep ivip 2021). The shift towards software-defined cars drives the increased usage of simulation in the automotive industry, especially for the validation of automated driving functions, which are ultimately used in rather complex, sociotechnical systems such as the traffic system (Stadler et al 2022;Hosse et al 2013;Johnson 2001;Harper et al 2021). As simulation models approximate only specific system behaviors to a limited scope, the question of trust also occurs with respect to the gap that can be accepted for virtual testing and modeling, especially when a direct comparison to the real world is impossible (Stadler et al 2022).…”

“…The shift towards software-defined cars drives the increased usage of simulation in the automotive industry, especially for the validation of automated driving functions, which are ultimately used in rather complex, sociotechnical systems such as the traffic system (Stadler et al 2022;Hosse et al 2013;Johnson 2001;Harper et al 2021). As simulation models approximate only specific system behaviors to a limited scope, the question of trust also occurs with respect to the gap that can be accepted for virtual testing and modeling, especially when a direct comparison to the real world is impossible (Stadler et al 2022). However, the fact that virtualized product development is not yet exploiting its full potential due to a lack of trust in M&S results does not mean that the domains and companies concerned, as well as the respective approval authorities responsible, have no interest in changing this situation.…”

RSG-GCN: Predicting Semantic Relationships in Urban Traffic Scene With Map Geometric Prior

“…The aim of the graph-preprocessing is to generate the semigraph (G semi = G AM ∪ G M M ∪ K AA ) used as our network's input. In this work, we use single-frame data to predict the semantic relationships between objects for the following reasons: (1) The RSG dataset we created has limited data and short videos of 20 seconds each. Using continuous frames like 5 seconds trajectory reduces the available dataset size by 75%.…”

“…This ratio will not decrease even with more annotations, as it is related to the nuScenes dataset's video length. Therefore, we prefer single-frame prediction to use more data for training;(2)Initially, we extended the node feature vector's dimension to include past positions for several time steps (1,3,5). However, experimental results showed poor performance, possibly due to the graph neural network model's sensitivity to the node feature vector dimension.…”

Validation of Parametric Opendrive Road Space Models

“…At the submicroscopic level, the individual vehicle, its subunits, and the interaction of the subunits with their environment are modeled and simulated (Hoogendoorn and Bovy, 2001). This level is particularly relevant in the vehicle development domain, as validated environment simulations of the system under test can substantially reduce the physical testing efforts and allow tests to be repeated in a reproducible manner (Stadler et al, 2022). Examples of submicroscopic driving simulators include IPG CarMaker, Vires VTD (von Neumann-Cosel, 2014), Tesis DYNA4 and CARLA (Dosovitskiy et al, 2017).…”