Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning

Qiao, Zhiqian; Tyree, Zachariah; Mudalige, Priyantha; Schneider, Jeff; Dolan, John M.

doi:10.1109/iros45743.2020.9341496

Cited by 23 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Concerning the violation of traffic rules, the work in [34] uses a penalty term for adversarial agents, with the work in [35] also adding a collision reward term as well as a penalty for unrealistic scenarios. Regarding safety distance, this has been considered by the authors in [36] who added a safety distance violation penalty and a collision penalty, among others, to a hierarchical RL model, by the authors in [37], who consider a fixed safety distance in overtaking maneuvers, and also by the authors in [38], where a distance reward is invoked in car-following maneuvers.…”

Section: Related Workmentioning

confidence: 99%

Increasing the Safety of Adaptive Cruise Control Using Physics-Guided Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

This paper presents a novel approach for improving the safety of vehicles equipped with Adaptive Cruise Control (ACC) by making use of Machine Learning (ML) and physical knowledge. More exactly, we train a Soft Actor-Critic (SAC) Reinforcement Learning (RL) algorithm that makes use of physical knowledge such as the jam-avoiding distance in order to automatically adjust the ideal longitudinal distance between the ego- and leading-vehicle, resulting in a safer solution. In our use case, the experimental results indicate that the physics-guided (PG) RL approach is better at avoiding collisions at any selected deceleration level and any fleet size when compared to a pure RL approach, proving that a physics-informed ML approach is more reliable when developing safe and efficient Artificial Intelligence (AI) components in autonomous vehicles (AVs).

show abstract

Section: Related Workmentioning

confidence: 99%

Increasing the Safety of Adaptive Cruise Control Using Physics-Guided Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Hierarchical Reinforcement Learning (HRL) can make the learning process more sample-efficient. The idea is to reuse the well trained network of one sub-goal on other similar tasks in HRL [17], [18].…”

Section: Reinforcement Learningmentioning

confidence: 99%

A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning

Sun

Chen

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Autonomous vehicles need to handle various traffic conditions and make safe and efficient decisions and maneuvers. However, on the one hand, a single optimization/samplingbased motion planner cannot efficiently generate safe trajectories in real time, particularly when there are many interactive vehicles near by. On the other hand, end-to-end learning methods cannot assure the safety of the outcomes. To address this challenge, we propose a hierarchical behavior planning framework with a set of low-level safe controllers and a high-level reinforcement learning algorithm (H-CtRL) as a coordinator for the low-level controllers. Safety is guaranteed by the low-level optimization/sampling-based controllers, while the high-level reinforcement learning algorithm makes H-CtRL an adaptive and efficient behavior planner. To train and test our proposed algorithm, we built a simulator that can reproduce traffic scenes using real-world datasets. The proposed H-CtRL is proved to be effective in various realistic simulation scenarios, with satisfying performance in terms of both safety and efficiency.

show abstract

“…To compute r nc , a dynamic distance (d r ) range is defined as a function of ego-vehicle's current speed (v ev ). If there exists a pedestrian crossing the road within (d r ), a penalty for the nearest front pedestrian is calculated using the following equations [17]:…”

Section: Reward Vectormentioning

confidence: 99%

Navigation in Urban Environments amongst pedestrians using Multi-Objective Deep Reinforcement Learning

Deshpande

Vaufreydaz

Spalanzani³

2021

2021 IEEE International Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

Urban autonomous driving in the presence of pedestrians as vulnerable road users is still a challenging and less examined research problem. This work formulates navigation in urban environments as a multi objective reinforcement learning problem. A deep learning variant of thresholded lexicographic Q-learning is presented for autonomous navigation amongst pedestrians. The multi objective DQN agent is trained on a custom urban environment developed in CARLA simulator. The proposed method is evaluated by comparing it with a single objective DQN variant on known and unknown environments. Evaluation results show that the proposed method outperforms the single objective DQN variant with respect to all aspects.

show abstract

Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning

Cited by 23 publications

References 12 publications

Increasing the Safety of Adaptive Cruise Control Using Physics-Guided Reinforcement Learning

Increasing the Safety of Adaptive Cruise Control Using Physics-Guided Reinforcement Learning

A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning

Navigation in Urban Environments amongst pedestrians using Multi-Objective Deep Reinforcement Learning

Contact Info

Product

Resources

About