Modeling Human Driving Behavior Through Generative Adversarial Imitation Learning

Bhattacharyya, Raunak P.; Wulfe, Blake; Phillips, D. C.; Kuefler, Alex; Morton, Jeremy; Senanayake, Ransalu; Kochenderfer, Mykel J.

doi:10.1109/tits.2022.3227738

Cited by 46 publications

(12 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Li et al (2023) employed the BC method to extract and replicate human driving skill parameters, enabling driving skill learning. Bhattacharyya et al (2023) utilized GAIL to extract skill parameters from different driving styles and achieve replication of various driving styles. Additionally, Kim et al (2020) introduced the Neural-Network-based Movement Primitive (NNMP) method, which models DMP using neural networks to retain task characteristics.…”

Section: Imitation Learning Methods For Robot Manipulation Tasksmentioning

confidence: 99%

Peg-in-hole assembly skill imitation learning method based on ProMPs under task geometric representation

Zang,

Wang,

Zha

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

IntroductionBehavioral Cloning (BC) is a common imitation learning method which utilizes neural networks to approximate the demonstration action samples for task manipulation skill learning. However, in the real world, the demonstration trajectories from human are often sparse and imperfect, which makes it challenging to comprehensively learn directly from the demonstration action samples. Therefore, in this paper, we proposes a streamlined imitation learning method under the terse geometric representation to take good advantage of the demonstration data, and then realize the manipulation skill learning of assembly tasks.MethodsWe map the demonstration trajectories into the geometric feature space. Then we align the demonstration trajectories by Dynamic Time Warping (DTW) method to get the unified data sequence so we can segment them into several time stages. The Probability Movement Primitives (ProMPs) of the demonstration trajectories are then extracted, so we can generate a lot of task trajectories to be the global strategy action samples for training the neural networks. Notalby, we regard the current state of the assembly task as the via point of the ProMPs model to get the generated trajectories, while the time point of the via point is calculated according to the probability model of the different time stages. And we get the action of the current state according to the target position of the next time state. Finally, we train the neural network to obtain the global assembly strategy by Behavioral Cloning.ResultsWe applied the proposed method to the peg-in-hole assembly task in the simulation environment based on Pybullet + Gym to test its task skill learning performance. And the learned assembly strategy was also executed on a real robotic platform to verify the feasibility of the method further.DiscussionAccording to the result of the experiment, the proposed method achieves higher success rates compared to traditional imitation learning methods while exhibiting reasonable generalization capabilities. It shows that the ProMPs under geometric representation can help the BC method make better use of the demonstration trajectory and thus better learn the task skills.

show abstract

Section: Imitation Learning Methods For Robot Manipulation Tasksmentioning

confidence: 99%

Peg-in-hole assembly skill imitation learning method based on ProMPs under task geometric representation

Zang,

Wang,

Zha

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…To improve driving (travel) efficiency and guarantee the completion of LC, terminal driving states are considered. Equation (16) shows the terminal cost for velocity, where v x (t h ) is the driving speed at time t h and V de is the desired velocity. Equation (17) formulates the terminal lateral position of ego vehicles to move from the current lane to the target lane or keep the current lane.…”

Section: Driving Comfort and Efficiencymentioning

confidence: 99%

“…A fuzzy logic approach was proposed to model both longitudinal and lateral driver behaviours, aiming to ensure integration between road safety and crash reduction based on an examination of speed limitations under weather conditions [15]. An integrated DBM was formulated as a sequential decision-making problem that is characterised by non-linearity and stochasticity, and unknown underlying cost functions, and an imitation learning approach optimised the formulation [16]. Previous DBM studies have demonstrated that the dynamic bicycle model is a robust description of vehicle dynamics.…”

Section: Introductionmentioning

confidence: 99%

A risk‐based driver behaviour model

Yuan,

Wang,

Calvert

et al. 2023

IET Intelligent Trans Sys

View full text Add to dashboard Cite

Current driver behaviour models (DBMs) are primarily designed for the general driver population under specific scenarios, such as car following or lane changing. Hence DBMs capturing individual behaviour under various scenarios are lacking. This paper presents a novel method to quantify individual perceived driving risk in the longitudinal and lateral directions using risk thresholds capturing the time headway and time to line crossing. These are integrated in a risk‐based DBM formulated under a model predictive control (MPC) framework taking into account vehicle dynamics. The DBM assumes drivers to operate as predictive controllers jointly optimising multiple criteria, including driving risk, discomfort, and travel inefficiency. Simulation results in car following and passing a slower vehicle demonstrate that the DBM predicts plausible behaviour under representative driving scenarios, and that the risk thresholds are able to reflect individual driving behaviour. Furthermore, the proposed DBM is verified using empirical driving data collected from a driving simulator, and the results show it is able to accurately generate vehicle longitudinal and lateral control matching individual human drivers. Overall, this model can capture individual risk perception behaviour and can be applied to the design and assessment of intelligent vehicle systems.

show abstract

“…They update adversarially in an unsupervised manner, potentially improving the deficiencies of supervised learning. Currently, in the field of longitudinal control for CAVs, car-following models based on GANs (Kuefler et al, 2017 ; Greveling, 2018 ; Zhou et al, 2020 ; Bhattacharyya et al, 2022 ; Mo and Di, 2022 ; Shi H. et al, 2022 ) are state-of-the-art. Nevertheless, few studies ponder over the continuity of time series.…”

Section: Introductionmentioning

confidence: 99%

Application of conditional generative adversarial network to multi-step car-following modeling

2023

Front. Neurorobot.

View full text Add to dashboard Cite

Car-following modeling is essential in the longitudinal control for connected and autonomous vehicles (CAVs). Considering the advantage of the generative adversarial network (GAN) in capturing realistic data distribution, this paper applies conditional GAN (CGAN) to car-following modeling. The generator is elaborately designed with a sequence-to-sequence structure to reflect the decision-making process of human driving behavior. The proposed model is trained and tested based on the empirical dataset, and it is compared with a supervised learning model and a mathematical model. Numerical simulations are conducted to verify the model's performance, especially in the condition of mixed traffic flow. The comparison result shows that the CGAN model outperforms others in trajectory reproduction, indicating it can effectively imitate human driving behavior. The simulation results suggest that the introduction of CGAN-based CAVs improves the stability and efficiency of the mixed traffic flow.

show abstract

Modeling Human Driving Behavior Through Generative Adversarial Imitation Learning

Cited by 46 publications

References 43 publications

Peg-in-hole assembly skill imitation learning method based on ProMPs under task geometric representation

Peg-in-hole assembly skill imitation learning method based on ProMPs under task geometric representation

A risk‐based driver behaviour model

Application of conditional generative adversarial network to multi-step car-following modeling

Contact Info

Product

Resources

About