Controlling an Autonomous Vehicle with Deep Reinforcement Learning

Folkers, Andreas; Rick, Matthias; Büskens, Christof

doi:10.1109/ivs.2019.8814124

Cited by 52 publications

(20 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep Reinforcement learning is nowadays the most popular technique for (artificial) agent to learn closely optimal strategy by experience. Majors companies are training self driving cars using reinforcement learning (see Folkers et al (2019), or Kiran et al (2020) for a state-of-the-art). Such techniques are extremely powerful to models behaviors of animals, consumers, investors, etc.…”

Section: Discussionmentioning

confidence: 99%

Reinforcement Learning in Economics and Finance

Charpentier¹,

Élie²,

Remlinger³

2020

Preprint

View full text Add to dashboard Cite

Reinforcement learning algorithms describe how an agent can learn an optimal action policy in a sequential decision process, through repeated experience. In a given environment, the agent policy provides him some running and terminal rewards. As in online learning, the agent learns sequentially. As in multi-armed bandit problems, when an agent picks an action, he can not infer ex-post the rewards induced by other action choices. In reinforcement learning, his actions have consequences: they influence not only rewards, but also future states of the world. The goal of reinforcement learning is to find an optimal policy -a mapping from the states of the world to the set of actions, in order to maximize cumulative reward, which is a long term strategy. Exploring might be sub-optimal on a short-term horizon but could lead to optimal long-term ones. Many problems of optimal control, popular in economics for more than forty years, can be expressed in the reinforcement learning framework, and recent advances in computational science, provided in particular by deep learning algorithms, can be used by economists in order to solve complex behavioral problems. In this article, we propose a state-of-the-art of reinforcement learning techniques, and present applications in economics, game theory, operation research and finance.

show abstract

Section: Discussionmentioning

confidence: 99%

Reinforcement Learning in Economics and Finance

Charpentier¹,

Élie²,

Remlinger³

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…6. The surrounding from the perspective of the vehicle can be described by a coarse perception map where the target is represented by a red dot (c) (source: [78]). since there is no reflection, which is provided by TORCS and used in [20], is to represent the lane markings with imagined beam sensors.…”

Section: E Observation Spacementioning

confidence: 99%

“…Though the previously cited examples didn't use RL techniques, they prove that grid representation holds high potential in this field. Navigation in a static environment by using a grid map as the observation, together with position and yaw of the vehicle with an RL agent, is presented in [78] (See Fig.6). Grid maps are also unstructured data, and their complexity is similar to the semantically segmented images, since the cells store class information in both cases, and hence their optimal handling is using the CNN architecture.…”

Section: E Observation Spacementioning

confidence: 99%

See 1 more Smart Citation

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

305

View full text Add to dashboard Cite

Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as carfollowing, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed.

show abstract

“…Reinforcement learning (RL) based methods usually use an end-to-end approach, trying to generate the direct steering, throttle, and brake commands based on the environmental information available. These researches use a various set of sensor models, such as grid-based topological [19], lidar-like beam sensors [20], camera information [21] or the high-level ground truth position information [22]. Another group of researches focus on strategic decisions, where the agent determines high-level actions, such as lane-change, follow, etc.…”

Section: A Related Workmentioning

confidence: 99%

Hierarchical Evasive Path Planning Using Reinforcement Learning and Model Predictive Control

2020

View full text Add to dashboard Cite

Motion planning plays an essential role in designing self-driving functions for connected and autonomous vehicles. The methods need to provide a feasible trajectory for the vehicle to follow, fulfilling different requirements, such as safety, efficiency, and passenger comfort. In this area, algorithms must also meet strict real-time expectations, since, especially in an emergency, the decision time is limited, which raises a trade-off for the feasibility requirements. This article proposes a hierarchical path planning solution for evasive maneuvering, where a Twin Delayed DDPG reinforcement learning agent generates the parameters of a geometric path consisting of chlotoids and straight sections, and an underlying model predictive control loop fulfills the trajectory following tasks. The method is applied to the automotive double lane-change test, a common emergency situation, comparing its results with human drivers' performance using a dynamic simulation environment. Besides the test's standardized parameters, a broader range of topological layouts is chosen, both for the training and performance evaluation. The results show that the proposed method highly outperforms human drivers, especially in challenging situations, while meeting the computational requirements, as the pre-trained neural network and path generation algorithm can provide a solution in an instant, based on the experience gained during the training process.

show abstract

Controlling an Autonomous Vehicle with Deep Reinforcement Learning

Cited by 52 publications

References 14 publications

Reinforcement Learning in Economics and Finance

Reinforcement Learning in Economics and Finance

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Hierarchical Evasive Path Planning Using Reinforcement Learning and Model Predictive Control

Contact Info

Product

Resources

About