Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator

Abstract

Intersections are considered one of the most complex scenarios in a self-driving framework due to the uncertainty in the behaviors of surrounding vehicles and the different types of scenarios that can be found. To deal with this problem, we provide a Deep Reinforcement Learning approach for intersection handling, which is combined with Curriculum Learning to improve the training process. The state space is defined by two vectors, containing adversaries and ego vehicle information. We define a features extractor module and an actor–critic approach combined with Curriculum Learning techniques, adding complexity to the environment by increasing the number of vehicles. In order to address a complete autonomous driving system, a hybrid architecture is proposed. The operative level generates the driving commands, the strategy level defines the trajectory and the tactical level executes the high-level decisions. This high-level decision system is the main goal of this research. To address realistic experiments, we set up three scenarios: intersections with traffic lights, intersections with traffic signs and uncontrolled intersections. The results of this paper show that a Proximal Policy Optimization algorithm can infer ego vehicle-desired behavior for different intersection scenarios based only on the behavior of adversarial vehicles.

Learn to Bet: Using Reinforcement Learning to Improve Vehicle Bids in Auction-Based Smart Intersections

“…Intersection Type Method [5] intersection manager unsignalized auctions (driver-specified budget) [23] intersection manager unsignalized auctions (two-step second-price sealed-bid) [25] intersection manager unsignalized reservation-based [29] intersection manager unsignalized reinforcement learning (to select predefined policy) [31] intersection manager signalized reinforcement learning [33] intersection manager unsignalized reinforcement learning [34] latencies unsignalized reinforcement learning [35] intersection manager signalized multi-agent reinforcement learning [36] vehicle manoeuvers unsignalized reinforcement learning [37] vehicle manoeuvers unsignalized and signalized reinforcement learning…”

“…The goal is to learn to break, steer, and throttle to safely cross the intersection. In [ 37 ], vehicles learn the desired policy to cross the intersection without any prior information about the scenario and infer both the intentions of the adversarial vehicles and the type of intersection.…”

Using Kolmogorov Entropy to Verify the Description Completeness of Traffic Dynamics of Highly Autonomous Driving

“…So far, very few projects have addressed the issue of completeness in the analysis of deep learning systems for sensing in highly autonomous vehicles [21,22]. The main reason for this is that the complexity required to determine the completeness of the system dynamics of a given traffic node could not be provided by conventional methods, and this requires an extension of the operations [23][24][25].…”

The Development of an Autonomous Vehicle Training and Verification System for the Purpose of Teaching Experiments

“…This approach allows for the avoidance of collisions with obstacles and enables the creation of optimized trajectories in a simulated environment. In addition, Gutiérrez-Moreno et al [16] presented an approach to intersection handling in autonomous driving, specifically the use of a deep reinforcement learning approach with curriculum learning and the effectiveness of the Proximal Policy Optimization algorithm in inferring desired behavior based on the behavior of adversarial vehicles in the CARLA simulator. Donkey Car, similar to the AWS DeepRacer, is a simulation system that provides both a virtual autonomous environment and physical vehicles.…”