Testing Autonomous Vehicle Software in the Virtual Prototyping Environment

Kim, Baekgyu; Kashiba, Yusuke; Dai, Siyuan; Shiraishi, Shinichi

doi:10.1109/les.2016.2644619

Cited by 41 publications

(24 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Beyond NN-focused testing and analysis, our work borrows ideas from robustness guided falsification for autonomous vehicles [35], where the goal is to detect boundary-case failures. Finally, our work falls under the broader context of automatic test generation methods for autonomous vehicles and driver assist systems [22,21], but our methods and testing goals are different.…”

Section: Related Workmentioning

confidence: 99%

Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components

Tuncali¹,

Fainekos

Ito³

et al. 2018

2018 IEEE Intelligent Vehicles Symposium (IV)

183

105

View full text Add to dashboard Cite

Many organizations are developing autonomous driving systems, which are expected to be deployed at a large scale in the near future. Despite this, there is a lack of agreement on appropriate methods to test, debug, and certify the performance of these systems. One of the main challenges is that many autonomous driving systems have machine learning components, such as deep neural networks, for which formal properties are difficult to characterize. We present a testing framework that is compatible with test case generation and automatic falsification methods, which are used to evaluate cyber-physical systems. We demonstrate how the framework can be used to evaluate closed-loop properties of an autonomous driving system model that includes the ML components, all within a virtual environment. We demonstrate how to use test case generation methods, such as covering arrays, as well as requirement falsification methods to automatically identify problematic test scenarios. The resulting framework can be used to increase the reliability of autonomous driving systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components

Tuncali¹,

Fainekos

Ito³

et al. 2018

2018 IEEE Intelligent Vehicles Symposium (IV)

183

105

View full text Add to dashboard Cite

show abstract

“…The only way to solve the problem is to continuously increase the accuracy and proportion of virtual testing through scientific methods based on closed environment and open road measured data [12]. But the virtual test model does inevitably simplify the real situation.…”

Section: Testing and Verificationmentioning

confidence: 99%

Analysis of Key Issues on Man-Control to System-Control Leap in Autonomous Driving

et al. 2019

View full text Add to dashboard Cite

Autonomous driving is recognized as a global development direction and a major opportunity. The function and use of the vehicle has changed profoundly. The vehicle is gradually transformed from a simple transportation tool to a smart mobile space. The ultimate goal of autonomous driving is to achieve driverless driving. In the course of its development, man-control will gradually turn into to system-control. In other words, the transition from level 3 (L3) to level 4 (L4) is a fundamental leap. At present, the specific path to achieve this leap is not yet clear. Different companies have different and even opposite thinking and choices. In this study, the grading standard for autonomous driving was clearly explained, and the technical route selection of the company was analysed. Based on the analysis, the requirements of sensing, decision making, execution between the L3 and L4 were compared. Moreover, the key technical difficulties of L3 to L4 were clarified. In the end, suggestions on the commercialization of autonomous driving were given.

show abstract

“…To demonstrate that automated vehicles have a better performance than humans with a 95% confidence level, they need to be driven for 440 million km [1]. Because this cannot be achieved through real-world testing alone, virtual tests are a common practice, making it possible to test many aspects faster than in real-time, see e.g., [2]- [4]. Even virtual tests, however, can be very time-consuming since dangerous or interesting situations are rare events.…”

Section: Introductionmentioning

confidence: 99%

Generating Critical Test Scenarios for Automated Vehicles with Evolutionary Algorithms

Klischat

Althoff

2019

2019 IEEE Intelligent Vehicles Symposium (IV)

116

View full text Add to dashboard Cite

Virtual testing of automated vehicles using simulations is essential during their development. When it comes to the testing of motion planning algorithms, one is mainly interested in challenging, critical scenarios for which it is hard to find a feasible solution. However, these situations are rare under usual traffic conditions, demanding an automatic generation of critical test scenarios. We present an approach that automatically generates critical scenarios based on a minimization of the solution space of the vehicle under test. By formulating a scenario parametrization and automatic determination of relevant parameter intervals, we are able to optimize the criticality of complex scenarios. We use evolutionary algorithms to tackle the resulting highly nonlinear optimization problem. Compared to our previous approach, we are now able to handle complex situations, in particular those involving intersections. Finally, we demonstrate our approach by generating critical scenarios from initially uncritical scenarios.All authors are with the Technische Universität München,

show abstract

Testing Autonomous Vehicle Software in the Virtual Prototyping Environment

Cited by 41 publications

References 3 publications

Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components

Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components

Analysis of Key Issues on Man-Control to System-Control Leap in Autonomous Driving

Generating Critical Test Scenarios for Automated Vehicles with Evolutionary Algorithms

Contact Info

Product

Resources

About