Modeling Human Driving Behavior through Generative Adversarial Imitation Learning

Bhattacharyya, Raunak P.; Wulfe, Blake; Phillips, Derek L.; Kuefler, Alex; Morton, Jeremy R.; Senanayake, Ransalu; Kochenderfer, Mykel J.

doi:10.48550/arxiv.2006.06412

Cited by 8 publications

(12 citation statements)

References 31 publications

(43 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GAIL was expanded to perform multi-agent imitation learning in simulated environments (Song et al, 2018). GAIL has recently been tested with mixed levels of success on realworld highway driving scenarios (Bhattacharyya, Phillips, et al, 2018;Bhattacharyya, Wulfe, et al, 2020). GAIL has not yet been implemented in more complex driving environments (e.g.…”

Section: Introductionmentioning

confidence: 99%

Multi-Vehicle Control in Roundabouts using Decentralized Game-Theoretic Planning

Jamgochian¹,

Menda²,

Kochenderfer³

2022

Preprint

View full text Add to dashboard Cite

Safe navigation in dense, urban driving environments remains an open problem and an active area of research. Unlike typical predict-thenplan approaches, game-theoretic planning considers how one vehicle's plan will affect the actions of another. Recent work has demonstrated significant improvements in the time required to find local Nash equilibria in general-sum games with nonlinear objectives and constraints. When applied trivially to driving, these works assume all vehicles in a scene play a game together, which can result in intractable computation times for dense traffic. We formulate a decentralized approach to game-theoretic planning by assuming that agents only play games within their observational vicinity, which we believe to be a more reasonable assumption for human driving. Games are played in parallel for all strongly connected components of an interaction graph, significantly reducing the number of players and constraints in each game, and therefore the time required for planning. We demonstrate that our approach can achieve collision-free, efficient driving in urban environments by comparing performance against an adaptation of the Intelligent Driver Model and centralized game-theoretic planning when navigating roundabouts in the INTER-ACTION dataset. Our implementation is available at http://github.com/sisl/DecNashPlanning.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Vehicle Control in Roundabouts using Decentralized Game-Theoretic Planning

Jamgochian¹,

Menda²,

Kochenderfer³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This inaccuracy may not affect significantly for the traffic flow analysis, but will cause significant evaluation biasedness on AV performance. To incorporate the external noise term (t), the most commonly used one is the Gaussian noise (Laval et al, 2014;He et al, 2015;Treiber and Kesting, 2017;Kuefler et al, 2017;Bhattacharyya et al, 2020). For example, Laval et al (2014) added an external Gaussian noise to Newell's car-following model (Newell, 2002) and showed that the stochastic car-following model can produce traffic oscillations that accord well with the observations.…”

Section: Introductionmentioning

confidence: 99%

Distributionally Consistent Simulation of Naturalistic Driving Environment for Autonomous Vehicle Testing

Yan¹,

Feng²,

Sun³

et al. 2021

Preprint

View full text Add to dashboard Cite

Microscopic traffic simulation provides a controllable, repeatable, and efficient testing environment for autonomous vehicles (AVs). To evaluate AVs' safety performance unbiasedly, ideally, the probability distributions of the joint state space of all vehicles in the simulated naturalistic driving environment (NDE) needs to be consistent with those from the real-world driving environment. However, although human driving behaviors have been extensively investigated in the transportation engineering field, most existing models were developed for traffic flow analysis without consideration of distributional consistency of driving behaviors, which may cause significant evaluation biasedness for AV testing. To fill this research gap, a distributionally consistent NDE modeling framework is proposed. Using large-scale naturalistic driving data, empirical distributions are obtained to construct the stochastic human driving behavior models under different conditions, which serve as the basic behavior models. To reduce the model errors caused by the limited data quantity and mitigate the error accumulation problem during the simulation, an optimization framework is designed to further enhance the basic models. Specifically, the vehicle state evolution is modeled as a Markov chain and its stationary distribution is twisted to match the distribution from the real-world driving environment. In the case study of highway driving environment using realworld naturalistic driving data, the distributional accuracy of the generated NDE is validated. The generated NDE is further utilized to test the safety performance of an AV model to validate its effectiveness.

show abstract

“…For instance, as humans, we know that vehicles should not collide with each other. However, as found by Bhattacharyya et al [10], it is challenging for completely data-driven techniques to learn such rules. Second, because the datadriven models are not interpretable, it is difficult to verify and validate them, making them less attractive for safetycritical applications such as autonomous driving.…”

Section: Introductionmentioning

confidence: 99%

“…Behavioral cloning [21] is one way to learn such demonstrations from data. However, such supervised learning techniques have proven to be less successful in applications such as modeling multiagent traffic due to compounding errors and not taking into account multi-agent interactions [10]. Techniques such as inverse reinforcement learning [22], [23] and inverse reward design [24] attempt to directly model the underlying reward function of humans.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Hybrid Rule-Based and Data-Driven Approach to Driver Modeling through Particle Filtering

Bhattacharyya¹,

Jung²,

Kruse³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous vehicles need to model the behavior of surrounding human driven vehicles to be safe and efficient traffic participants. Existing approaches to modeling human driving behavior have relied on both data-driven and rulebased methods. While data-driven models are more expressive, rule-based models are interpretable, which is an important requirement for safety-critical domains like driving. However, rule-based models are not sufficiently representative of data, and data-driven models are yet unable to generate realistic traffic simulation due to unrealistic driving behavior such as collisions. In this paper, we propose a methodology that combines rule-based modeling with data-driven learning. While the rules are governed by interpretable parameters of the driver model, these parameters are learned online from driving demonstration data using particle filtering. We perform driver modeling experiments on the task of highway driving and merging using data from three real-world driving demonstration datasets. Our results show that driver models based on our hybrid rule-based and data-driven approach can accurately capture real-world driving behavior. Further, we assess the realism of the driving behavior generated by our model by having humans perform a "driving Turing test," where they are asked to distinguish between videos of real driving and those generated using our driver models.

show abstract

Modeling Human Driving Behavior through Generative Adversarial Imitation Learning

Cited by 8 publications

References 31 publications

Multi-Vehicle Control in Roundabouts using Decentralized Game-Theoretic Planning

Multi-Vehicle Control in Roundabouts using Decentralized Game-Theoretic Planning

Distributionally Consistent Simulation of Naturalistic Driving Environment for Autonomous Vehicle Testing

A Hybrid Rule-Based and Data-Driven Approach to Driver Modeling through Particle Filtering

Contact Info

Product

Resources

About