A Survey of Deep RL and IL for Autonomous Driving Policy Learning

Abstract

Autonomous driving (AD) agents generate driving policies based on online perception results, which are obtained at multiple levels of abstraction, e.g., behavior planning, motion planning and control. Driving policies are crucial to the realization of safe, efficient and harmonious driving behaviors, where AD agents still face substantial challenges in complex scenarios. Due to their successful application in fields such as robotics and video games, the use of deep reinforcement learning (DRL) and deep imitation learning (DIL) techniques to derive AD policies have witnessed vast research efforts in recent years. This paper is a comprehensive survey of this body of work, which is conducted at three levels: First, a taxonomy of the literature studies is constructed from the system perspective, among which five modes of integration of DRL/DIL models into an AD architecture are identified. Second, the formulations of DRL/DIL models for conducting specified AD tasks are comprehensively reviewed, where various designs on the model state and action spaces and the reinforcement learning rewards are covered. Finally, an in-depth review is conducted on how the critical issues of AD applications regarding driving safety, interaction with other traffic participants and uncertainty of the environment are addressed by the DRL/DIL models. To the best of our knowledge, this is the first survey to focus on AD policy learning using DRL/DIL, which is addressed simultaneously from the system, task-driven and problem-driven perspectives. We share and discuss findings, which may lead to the investigation of various topics in the future.

Arena-Bench: A Benchmarking Suite for Obstacle Avoidance Approaches in Highly Dynamic Environments

“…A common barrier is that most of the research work evaluated their approaches on specifically designed simulation environments or test setups, making a general comparison against existing approaches difficult [5]. Furthermore, deployment and integration of DRL into real robotic platforms is still an open frontier due to safety reasons [6], [7]. Thus, a benchmark to properly assess those approaches in realistic scenarios and against existing algorithms is not only an essential step towards the deployment of DRL into real robots but also assists in the development and validation of learning-based approaches on mobile robots.…”

Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving

“…A majority of these studies perform both the training and evaluation in simulated environments, whereas some train the agent in simulations and then apply the trained agent in the real world [9], [10], or for some limited cases, the training itself is also performed in the real world [11]. Overviews of RL for autonomous driving are given by Kiran et al [12] and by Zhu et al [13]. However, previous studies do not estimate the aleatoric or the epistemic uncertainty of the decision that the trained agent recommends.…”