Experience Paper: Search-Based Testing in Automated Driving Control Applications

Gladisch, Christoph; Heinz, Thomas; Heinzemann, Christian; Oehlerking, Jens; Vietinghoff, Anne von; Pfitzer, Tim

doi:10.1109/ase.2019.00013

Cited by 50 publications

(22 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In comparison, we consider the closed-loop behavior of the system and our parameters directly change the world rather than apply transformations post facto. We can observe, over time, that certain vehicles are not detected, which is more useful to testers than a single misclassification [31]. Furthermore, it is already known that structural coverage criteria may not be an effective strategy for finding errors in classification [39].…”

Section: Related Workmentioning

confidence: 95%

“…For instance, in the following example, we check that there is no collision and ensure that the collision was not trivially avoided because our vehicle did not move at all. The ability to write monitors which read streams of system-generated events provides an expressive framework to write temporal properties, something that has been identified as a major limitation of prior tools [31]. Monitors for metric and signal temporal logic specifications can be encoded in the usual way [18,33].…”

Section: Autonomous Vehicles and System Under Test (Sut)mentioning

confidence: 99%

“…This makes the setup of varied scenarios difficult and time consuming. Though exploiting parametricity in simulation is useful and effective [10,23,31,67], the cost of environment setup, and navigating large parameter spaces, is quite high [31]. Prior works have used bespoke environments with limited parametricity.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Paracosm: A Test Framework for Autonomous Driving Simulations

Majumdar

Mathur

Pirron

et al. 2021

Fundamental Approaches to Software Engineering

View full text Add to dashboard Cite

Systematic testing of autonomous vehicles operating in complex real-world scenarios is a difficult and expensive problem. We present Paracosm, a framework for writing systematic test scenarios for autonomous driving simulations. Paracosm allows users to programmatically describe complex driving situations with specific features, e.g., road layouts and environmental conditions, as well as reactive temporal behaviors of other cars and pedestrians. A systematic exploration of the state space, both for visual features and for reactive interactions with the environment is made possible. We define a notion of test coverage for parameter configurations based on combinatorial testing and low dispersion sequences. Using fuzzing on parameter configurations, our automatic test generator can maximize coverage of various behaviors and find problematic cases. Through empirical evaluations, we demonstrate the capabilities of Paracosm in programmatically modeling parameterized test environments, and in finding problematic scenarios.

show abstract

Section: Related Workmentioning

confidence: 95%

Section: Autonomous Vehicles and System Under Test (Sut)mentioning

confidence: 99%

See 1 more Smart Citation

Paracosm: A Test Framework for Autonomous Driving Simulations

Majumdar

Mathur

Pirron

et al. 2021

Fundamental Approaches to Software Engineering

View full text Add to dashboard Cite

show abstract

“…In this example, our goal is to create a scene for Fetch that resembles a pick and place playground. 4 Throughout the rest of this section, we explain how GzScenic achieves this goal. As discussed in Section II-A, parsing a scenario in Scenic language and generating a valid concrete scene requires a set of model definitions that should be provided to Scenic.…”

Section: Gzscenicmentioning

confidence: 99%

GzScenic: Automatic Scene Generation for Gazebo Simulator

Afzal,

Goues,

Timperley

2021

Preprint

View full text Add to dashboard Cite

Testing robotic and cyberphysical systems in simulation require specifications of the simulated environments (i.e., scenes). The Scenic domain-specific language provides a high-level probabilistic programming language that allows users to specify scenarios for simulation. Scenic automatically generates concrete scenes that can be rendered by simulators. However, Scenic is mainly designed for autonomous vehicle simulation and does not support the most popular generalpurpose simulator: Gazebo. In this work, we present GzScenic; a tool that automatically generates scenes for simulation in Gazebo. GzScenic automatically generates both the models required for running Scenic on the scenarios, and the models that Gazebo requires for running the simulation.

show abstract

“…In verification, software components are typically tested against their functional requirements in scenarios defined by formal specifications. Works on falsification aim at falsifying planning algorithms of automated vehicles against a given logic formula, e.g., formulated in signal temporal logic (STL) [7], [8]. However, these works do not answer the question of how to ensure that the behavior of surrounding vehicles conforms to assumptions the test specification is based on.…”

Section: A Related Workmentioning

confidence: 99%

Synthesizing Traffic Scenarios from Formal Specifications for Testing Automated Vehicles

Klischat

Althoff

2020

2020 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Virtual testing plays an important role in the validation and verification of automated vehicles. State-of-theart approaches first generate a huge amount of test scenarios through simulations or test drives, which are later filtered to obtain relevant scenarios for a given set of specifications. However, only few works exist on synthesizing scenarios directly from specifications. In this work, we present an optimizationbased approach to synthesize scenarios only from formal specifications and a given map. To concretize the specifications, we formulate predicates, which are subsequently converted to a mixed-integer quadratic optimization problem. We demonstrate how our method can generate scenarios for maps featuring merging lanes and intersections given a variety of specifications.

show abstract

Experience Paper: Search-Based Testing in Automated Driving Control Applications

Cited by 50 publications

References 28 publications

Paracosm: A Test Framework for Autonomous Driving Simulations

Paracosm: A Test Framework for Autonomous Driving Simulations

GzScenic: Automatic Scene Generation for Gazebo Simulator

Synthesizing Traffic Scenarios from Formal Specifications for Testing Automated Vehicles

Contact Info

Product

Resources

About