A Sensitivity Analysis Approach for Evaluating a Radar Simulation for Virtual Testing of Autonomous Driving Functions

Ngo, Anthony; Bauer, Max Paul; Resch, Michael

doi:10.1109/acirs49895.2020.9162598

Cited by 11 publications

(8 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [96], a sensitivity analysis is performed to determine the most critical aspects that contribute to realistic RADAR simulation. The analysis is carried out by comparing real sensor data vs. synthetic ones generated using the opensource CARLA simulator [97].…”

Section: ) Sensor Modelsmentioning

confidence: 99%

Virtual Testing of Automated Driving Systems. A Survey on Validation Methods

Donà¹,

Ciuffo

2022

IEEE Access

View full text Add to dashboard Cite

This paper surveys the state-of-the-art contributions supporting the validation of virtual testing toolchains for Automated Driving System (ADS) verification. The work builds upon the wellknown limitations of physical testing while conceiving the virtual counterpart as a fundamental ingredient for the type-approval of high automation level ADS. The purpose of the research effort is to summarize computational tools, validation methodologies, and the corresponding fidelity levels delivered by state-ofthe-art simulation toolchains. The ultimate goal is to establish how effectively simulation can play the role of a "virtual proving ground" for ADS certification independently from any specific ADS implementation/effectiveness. The contribution includes classic high-level validation approaches and modern specific computational tools that can be adopted depending on the type of data under analysis. Moreover, the investigation covers approaches embraced both within the scientific community and in technical regulations for the sake of completeness. Ultimately, we identified two high-level validation schema: integrated environment and subsystem-based solutions. In addition, we found that modeling and validating virtual sensors for ADS is the most lacking area from a subsystem-level approach. On the other side, the closedloop interaction between the ADS and other virtual traffic participants makes it difficult to directly compare the experimental results with simulated generated evidence as the emergent behaviors of the ADS may amplify minor discrepancies between the environments.

show abstract

Section: ) Sensor Modelsmentioning

confidence: 99%

Virtual Testing of Automated Driving Systems. A Survey on Validation Methods

Donà¹,

Ciuffo

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Ngo et al [ 42 ] distinguish time-domain electromagnetic simulation techniques, ray tracing, data-driven, and idealised modelling approaches. They describe a method for evaluating a radar sensor model by comparing the results of clustering algorithms with real and synthetic radar data and provide a sensitivity analysis for the various parameters of their radar sensor model.…”

Section: State-of-the-art Radar Sensor Model Classificationsmentioning

confidence: 99%

A Survey on Modelling of Automotive Radar Sensors for Virtual Test and Validation of Automated Driving

Magosi

Rosenberger

et al. 2022

Sensors

View full text Add to dashboard Cite

Radar sensors were among the first perceptual sensors used for automated driving. Although several other technologies such as lidar, camera, and ultrasonic sensors are available, radar sensors have maintained and will continue to maintain their importance due to their reliability in adverse weather conditions. Virtual methods are being developed for verification and validation of automated driving functions to reduce the time and cost of testing. Due to the complexity of modelling high-frequency wave propagation and signal processing and perception algorithms, sensor models that seek a high degree of accuracy are challenging to simulate. Therefore, a variety of different modelling approaches have been presented in the last two decades. This paper comprehensively summarises the heterogeneous state of the art in radar sensor modelling. Instead of a technology-oriented classification as introduced in previous review articles, we present a classification of how these models can be used in vehicle development by using the V-model originating from software development. Sensor models are divided into operational, functional, technical, and individual models. The application and usability of these models along the development process are summarised in a comprehensive tabular overview, which is intended to support future research and development at the vehicle level and will be continuously updated.

show abstract

“…In the context of testing AVs, SA are commonly applied to find sensitivities of model parameters in AV sub-modules. It has been applied to assess which parameters of a radar simulation have the greatest impact on the performance of a spatial clustering algorithm which is applied to the simulated output of the radar models [16]. Vehicle dynamics models have also been analysed towards their sensitivities, as the quality of the used vehicle dynamics models has an effect on AV simulations [17].…”

Section: Scenario-based Validation Of Avsmentioning

confidence: 99%

Scenario Optimisation and Sensitivity Analysis for Safe Automated Driving Using Gaussian Processes

et al. 2021

View full text Add to dashboard Cite

Assuring the safety of automated vehicles is essential for their timely introduction and acceptance by policymakers and the public. To assess their safe design and robust decision making in response to all possible scenarios, new methods that use a scenario-based testing approach are needed, as testing on public roads in normal traffic would require driving millions of kilometres. We make use of the scenario-based testing approach and propose a method to model simulated scenarios using Gaussian Process based models to predict untested scenario outcomes. This enables us to efficiently determine the performance boundary, where the safe and unsafe scenarios can be evidently distinguished from each other. We present an iterative method that optimises the parameter space of a logical scenario towards the most critical scenarios on this performance boundary. Additionally, we conduct a novel probabilistic sensitivity analysis by efficiently computing several variance-based sensitivity indices using the Gaussian Process models and evaluate the relative importance of the scenario input parameters on the scenario outcome. We critically evaluate and investigate the usefulness of the proposed Gaussian Process based approach as a very efficient surrogate model, which can model the logical scenarios effectively in the presence of uncertainty. The proposed approach is applied on an exemplary logical scenario and shows viability in finding concrete critical scenarios. The reported results, derived from the proposed approach, could pave the way to more efficient testing of automated vehicles and instruct further physical tests on the determined critical scenarios.

show abstract

A Sensitivity Analysis Approach for Evaluating a Radar Simulation for Virtual Testing of Autonomous Driving Functions

Cited by 11 publications

References 25 publications

Virtual Testing of Automated Driving Systems. A Survey on Validation Methods

Virtual Testing of Automated Driving Systems. A Survey on Validation Methods

A Survey on Modelling of Automotive Radar Sensors for Virtual Test and Validation of Automated Driving

Scenario Optimisation and Sensitivity Analysis for Safe Automated Driving Using Gaussian Processes

Contact Info

Product

Resources

About