Deep reinforcement learning based control for Autonomous Vehicles in CARLA

Abstract

Nowadays, Artificial Intelligence (AI) is growing by leaps and bounds in almost all fields of technology, and Autonomous Vehicles (AV) research is one more of them. This paper proposes the using of algorithms based on Deep Learning (DL) in the control layer of an autonomous vehicle. More specifically, Deep Reinforcement Learning (DRL) algorithms such as Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG) are implemented in order to compare results between them. The aim of this work is to obtain a trained model, applying a DRL algorithm, able of sending control commands to the vehicle to navigate properly and efficiently following a determined route. In addition, for each of the algorithms, several agents are presented as a solution, so that each of these agents uses different data sources to achieve the vehicle control commands. For this purpose, an open-source simulator such as CARLA is used, providing to the system with the ability to perform a multitude of tests without any risk into an hyper-realistic urban simulation environment, something that is unthinkable in the real world. The results obtained show that both DQN and DDPG reach the goal, but DDPG obtains a better performance. DDPG perfoms trajectories very similar to classic controller as LQR. In both cases RMSE is lower than 0.1m following trajectories with a range 180-700m. To conclude, some conclusions and future works are commented.

Hybrid Traffic Control and Coordination from Pixels

“…A significant portion of training RVs via RL focuses on individual vehicle driving but not controlling the entire traffic [18,19]. To list some examples, images are used with vision transformers to learn an effective driving policy [20], to train RVs to drive in simulation [21,22,23,24], or to prevent crashes by capturing the RVs' surroundings [25]. In the work by Pan et al [26], simulated images are transformed into realistic-looking images to train RVs via RL.…”

Analyzing the inconsistency in driving patterns between manual and autonomous modes under complex driving scenarios with a VR-enabled simulation platform

“…Over recent years, CARLA by Intel (Dosovitskiy et al , 2017) and Drive Constellation by NVIDIA (NVIDIA, 2017) are convenient tools for AV testing. For example, they are used to examine the functions of different sensors (Rosique et al , 2019) and preview certain AV performances (Pérez-Gil et al , 2022) in naturalistic environments. While these types of tools are proved to be powerful, the test scenes are usually fixed or predefined.…”

Dynamic Goal Tracking for Differential Drive Robot using Deep Reinforcement Learning

“…Extensively used on unmanned aerial vehicles (UAVs) [10], covering legged robots of OpenAI Gym environments including ant, half cheetah and walker [8]. Moreover, fewer implementations of DDPG on differential drives are available in literature, e.g., autonomous driving cars [11], skid steering in differential drive [12], optimal torque distribution [13], and obstacle detection for differential drive [14]. However, implementation of TD3 on differential drive to the best of our knowledge has not been done.…”