Fitness Functions for Testing Automated and Autonomous Driving Systems

Hauer, Florian; Pretschner, Alexander; Holzmüller, Bernd

doi:10.1007/978-3-030-26601-1_5

Cited by 44 publications

(25 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Database: All primary studies referring to the database in this cluster employ it to analyze parameter distribution as prior knowledge. Based on our findings, we cluster the database into three types: naturalistic driving data [86], [87], [88], [106], traffic flow data [53], [79] and collision accident data [104]. It is worth noting that NDS databases exclusive to autonomous driving (e.g., Safety Pool) are available but have not been found applied in the field of this cluster.…”

Section: Required Informationmentioning

confidence: 99%

Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review

Zhang¹,

Tao²,

Tan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Scenario-based approaches have been receiving a huge amount of attention in research and engineering of automated driving systems. Due to the complexity and uncertainty of the driving environment, and the complexity of the driving task itself, the number of possible driving scenarios that an ADS or ADAS may encounter is virtually infinite. Therefore it is essential to be able to reason about the identification of scenarios and in particular critical ones that may impose unacceptable risk if not considered. Critical scenarios are particularly important to support design, verification and validation efforts, and as a basis for a safety case. In this paper, we present the results of a systematic literature review in the context of autonomous driving. The main contributions are: (i) introducing a comprehensive taxonomy for critical scenario identification methods; (ii) giving an overview of the state-of-the-art research based on the taxonomy encompassing 86 papers between 2017 and 2020; and (iii) identifying open issues and directions for further research. The provided taxonomy comprises three main perspectives encompassing the problem definition (the why), the solution (the methods to derive scenarios), and the assessment of the established scenarios. In addition, we discuss open research issues considering the perspectives of coverage, practicability, and scenario space explosion.

show abstract

Section: Required Informationmentioning

confidence: 99%

Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review

Zhang¹,

Tao²,

Tan³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A wide array of studies focus on applying traditional testing techniques to AVs including adaptive stress testing [40], where noise is injected into the input sensors of an AV to cause accidents; fitness function templates for testing automated and autonomous driving systems with heuristic search [54]; and search-based optimization [62]. These studies provide limited insights into the testing of realworld AVs, since they do not evaluate their techniques on opensource, production-grade AV software.…”

Section: Related Workmentioning

confidence: 99%

scenoRITA: Generating Less-Redundant, Safety-Critical and Motion Sickness-Inducing Scenarios for Autonomous Vehicles

Almanee¹,

Wu²,

Huai³

et al. 2021

Preprint

View full text Add to dashboard Cite

There is tremendous global enthusiasm for research, development, and deployment of autonomous vehicles (AVs), e.g., selfdriving taxis and trucks from Waymo and Baidu. The current practice for testing AVs uses virtual tests-where AVs are tested in software simulations-since they offer a more efficient and safer alternative compared to field operational tests. Specifically, searchbased approaches are used to find particularly critical situations. These approaches provide an opportunity to automatically generate tests; however, systematically creating valid and effective tests for AV software remains a major challenge. To address this challenge, we introduce scenoRITA, a test generation approach for AVs that uses evolutionary algorithms with (1) a novel gene representation that allows obstacles to be fully mutable, hence, resulting in more reported violations, (2) 5 test oracles to determine both safety and motion sickness-inducing violations, and (3) a novel technique to identify and eliminate duplicate tests. Our extensive evaluation shows that scenoRITA can produce effective driving scenarios that expose an ego car to safety critical situations. scenoRITA generated tests that resulted in a total of 1,026 unique violations, increasing the number of reported violations by 23.47% and 24.21% compared to random test generation and state-of-the-art partially-mutable test generation, respectively.

show abstract

“…Criticality metrics are combined with the occurrence rates to efficiently sample critical scenarios representative of real-world driving conditions [24]. Other approaches use optimization to create critical scenarios based on these metrics [25], [26]. Similarly, falsification methods can detect scenarios that falsify a motion planner with respect to a given safety specification [27], [28].…”

Section: A Related Workmentioning

confidence: 99%

Scenario Factory: Creating Safety-Critical Traffic Scenarios for Automated Vehicles

Klischat

Liu

Holtke

et al. 2020

2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

The safety validation of motion planning algorithms for automated vehicles requires a large amount of data for virtual testing. Currently, this data is often collected through real test drives, which is expensive and inefficient, given that only a minority of traffic scenarios pose challenges to motion planners. We present a workflow for generating a database of challenging and safety-critical test scenarios that is not dependent on recorded data. First, we extract a large variety of road networks across the globe from OpenStreetMap. Subsequently, we generate traffic scenarios for these road networks using the traffic simulator SUMO. In the last step, we increase the criticality of these scenarios using nonlinear optimization. Our generated scenarios are publicly available on the CommonRoad website.

show abstract

Fitness Functions for Testing Automated and Autonomous Driving Systems

Cited by 44 publications

References 14 publications

Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review

Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review

scenoRITA: Generating Less-Redundant, Safety-Critical and Motion Sickness-Inducing Scenarios for Autonomous Vehicles

Scenario Factory: Creating Safety-Critical Traffic Scenarios for Automated Vehicles

Contact Info

Product

Resources

About