Autonomous Driving in Roundabout Maneuvers Using Reinforcement Learning with Q-Learning

Cuenca, Laura García; Puertas, Enrique; Andrés, Javier Fernández; Aliane, Nourdine

doi:10.3390/electronics8121536

Cited by 46 publications

(23 citation statements)

References 31 publications

(35 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The presented dataset may be helpful for generating knowledge using machine learning techniques carrying out driving pattern classification in roundabouts, and for predicting vehicle speed and steering wheel in the different sections of roundabouts, as shown in [ 19 ], where algorithms such as Support Vector Machine, Lineal Regression, and Deep Learning are used to obtain different predictive data models. Other machine learning techniques that can be used on this autonomous driving dataset are algorithms based on reinforcement learning as in [ 6 ], where a Markov decision process (MDP) was used to study the behavior of a vehicle in order to safely navigate roundabouts using the Q-learning algorithm in a simulation environment. Regarding future works, it is planned to upgrade the built dataset and to apply the same approach to generate similar datasets driving in the urban intersection and highway entrances.…”

Section: Discussionmentioning

confidence: 99%

“…Automated driving implements a driver model transforming information perceived from real-world sensor measurements into actions on the vehicle’s actuators, such as steering wheel or pedals [ 2 ]. However, autonomous driving is a challenging task, and it is even more complex in dynamic environments, such as roundabouts or intersections, among others [ 3 , 4 , 5 , 6 , 7 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dataset Construction from Naturalistic Driving in Roundabouts

Cuenca

Guindel

Aliane

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

A proper driver characterization in complex environments using computational techniques depends on the richness and variety of data obtained from naturalistic driving. The present article proposes the construction of a dataset from naturalistic driving specific to maneuvers in roundabouts and makes it open and available to the scientific community for performing their own studies. The dataset is a combination of data gathered from on-board instrumentation and data obtained from the post-processing of maps as well as recorded videos. The approach proposed in this paper consists of handling roundabouts as a stretch of road that includes 100 m before the entrance, the internal part, and 100 m after the exit. This stretch of road is then spatially sampled in small sections to which data are associated.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Dataset Construction from Naturalistic Driving in Roundabouts

Cuenca

Guindel

Aliane

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…To overcome this issue, a partially observable MDP (POMDP) was applied to keep a probability distribution through a set of observations [22]. Recent studies have used RL for transportation issues-namely, adaptive traffic signal control [23,24] and autonomous vehicle agents in roundabouts [25]. Furthermore, the development of a deep neural network (DNN) can enhance feature extraction representations for complex tasks based on multi-hidden layers.…”

Section: Introductionmentioning

confidence: 99%

An Efficiency Enhancing Methodology for Multiple Autonomous Vehicles in an Urban Network Adopting Deep Reinforcement Learning

Tran

Bae

2021

Applied Sciences

View full text Add to dashboard Cite

To reduce the impact of congestion, it is necessary to improve our overall understanding of the influence of the autonomous vehicle. Recently, deep reinforcement learning has become an effective means of solving complex control tasks. Accordingly, we show an advanced deep reinforcement learning that investigates how the leading autonomous vehicles affect the urban network under a mixed-traffic environment. We also suggest a set of hyperparameters for achieving better performance. Firstly, we feed a set of hyperparameters into our deep reinforcement learning agents. Secondly, we investigate the leading autonomous vehicle experiment in the urban network with different autonomous vehicle penetration rates. Thirdly, the advantage of leading autonomous vehicles is evaluated using entire manual vehicle and leading manual vehicle experiments. Finally, the proximal policy optimization with a clipped objective is compared to the proximal policy optimization with an adaptive Kullback–Leibler penalty to verify the superiority of the proposed hyperparameter. We demonstrate that full automation traffic increased the average speed 1.27 times greater compared with the entire manual vehicle experiment. Our proposed method becomes significantly more effective at a higher autonomous vehicle penetration rate. Furthermore, the leading autonomous vehicles could help to mitigate traffic congestion.

show abstract

“…Moreover, the information includes the number of dynamic objects, presence of pedestrians, cyclists, motorcyclists and the distances between vehicles. Another solution based on intelligent methods is proposed in [8], where the vehicles are trained to cross a roundabout using the Q-learning algorithm. Furthermore, a solution based on fuzzy logic is proposed in [9] to compute the velocity for vehicles in order to ensure safe travel through the intersection and to decrease the emission of CO 2 .…”

Section: Introductionmentioning

confidence: 99%

Multivariable Optimisation for Waiting-Time Minimisation at Roundabout Intersections in a Cyber-Physical Framework

Pauca

Maxim

Caruntu

2021

Sensors

View full text Add to dashboard Cite

The evolution of communication networks offers new possibilities for development in the automotive industry. Smart vehicles will benefit from the possibility of connecting with the infrastructure and from an extensive exchange of data between them. Furthermore, new control strategies can be developed that benefit the advantages of these communication networks. In this endeavour, the main purposes considered by the automotive industry and researchers from academia are defined by: (i) ensuring people’s safety; (ii) reducing the overall costs, and (iii) improving the traffic by maximising the fluidity. In this paper, a cyber-physical framework (CPF) to control the access of vehicles in roundabout intersections composed of two levels is proposed. Both levels correspond to the cyber part of the CPF, while the physical part is composed of the vehicles crossing the roundabout. The first level, i.e., the edge-computing layer, is based on an analytical solution that uses multivariable optimisation to minimise the waiting times of the vehicles entering a roundabout intersection and to ensure a safe crossing. The second level, i.e., the cloud-computing layer, stores information about the waiting times and trajectories of all the vehicles that cross the roundabout and uses them for long-term analysis and prediction. The simulated results show the efficacy of the proposed method, which can be easily implemented on an embedded device for real-time operation.

show abstract

Autonomous Driving in Roundabout Maneuvers Using Reinforcement Learning with Q-Learning

Cited by 46 publications

References 31 publications

Dataset Construction from Naturalistic Driving in Roundabouts

Dataset Construction from Naturalistic Driving in Roundabouts

An Efficiency Enhancing Methodology for Multiple Autonomous Vehicles in an Urban Network Adopting Deep Reinforcement Learning

Multivariable Optimisation for Waiting-Time Minimisation at Roundabout Intersections in a Cyber-Physical Framework

Contact Info

Product

Resources

About