Design of Unsignalized Roundabouts Driving Policy of Autonomous Vehicles Using Deep Reinforcement Learning

Wang, Zengrong; Liu, Xujin; Wu, Zhifei

doi:10.3390/wevj14020052

Cited by 9 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Traffic signal control was initially set at fixed time intervals, but this fixed-time control cannot effectively adapt to changing traffic flows [21]. Therefore, the Adaptive Traffic Light Control (ATLC) technique has been developed to alleviate traffic congestion through the dynamic modification of signal timing.…”

Section: Related Workmentioning

confidence: 99%

A Comparative Study of Traffic Signal Control Based on Reinforcement Learning Algorithms

Ouyang,

Zhan,

2024

WEVJ

View full text Add to dashboard Cite

In recent years, the increasing production and sales of automobiles have led to a notable rise in congestion on urban road traffic systems, particularly at ramps and intersections with traffic signals. Intelligent traffic signal control represents an effective means of addressing traffic congestion. Reinforcement learning methods have demonstrated considerable potential for addressing complex traffic signal control problems with multidimensional states and actions. In this research, the team propose Q-learning and Deep Q-Network (DQN) based signal control frameworks that use variable phase sequences and cycle times to adjust the order and the duration of signal phases to obtain a stable traffic signal control strategy. Experiments are simulated using the traffic simulator Simulation of Urban Mobility (SUMO) to test the average speed and the lane occupancy rate of vehicles entering the ramp to evaluate its safety performance and test the vehicle’s traveling time to assess its stability. The simulation results show that both reinforcement learning algorithms are able to control cars in dynamic traffic environments with higher average speed and lower lane occupancy rate than the no-control method and that the DQN control model improves the average speed by about 10% and reduces the lane occupancy rate by about 30% compared to the Q-learning control model, providing a higher safety performance.

show abstract

Section: Related Workmentioning

confidence: 99%

A Comparative Study of Traffic Signal Control Based on Reinforcement Learning Algorithms

Ouyang,

Zhan,

2024

WEVJ

View full text Add to dashboard Cite

show abstract

“…For instance, García et al [13] proposed a method based on the Q-learning algorithm to train autonomous vehicle agents to navigate appropriately within roundabouts. Wang et al [14] introduced a driving strategy based on the Soft Actor-Critic (SAC) algorithm to ensure safety while minimizing costs. Zhang et al [15] employed optimization-embedded reinforcement learning (OERL) to achieve adaptive decision-making at roundabout intersections.…”

Section: Autonomous Driving Technology In Roundaboutsmentioning

confidence: 99%

Perception Enhanced Deep Deterministic Policy Gradient for Autonomous Driving in Complex Scenarios

Liao,

Xiao,

Xing

et al. 2024

CMES

View full text Add to dashboard Cite

Autonomous driving has witnessed rapid advancement; however, ensuring safe and efficient driving in intricate scenarios remains a critical challenge. In particular, traffic roundabouts bring a set of challenges to autonomous driving due to the unpredictable entry and exit of vehicles, susceptibility to traffic flow bottlenecks, and imperfect data in perceiving environmental information, rendering them a vital issue in the practical application of autonomous driving. To address the traffic challenges, this work focused on complex roundabouts with multi-lane and proposed a Perception Enhanced Deep Deterministic Policy Gradient (PE-DDPG) for Autonomous Driving in the Roundabouts. Specifically, the model incorporates an enhanced variational autoencoder featuring an integrated spatial attention mechanism alongside the Deep Deterministic Policy Gradient framework, enhancing the vehicle's capability to comprehend complex roundabout environments and make decisions. Furthermore, the PE-DDPG model combines a dynamic path optimization strategy for roundabout scenarios, effectively mitigating traffic bottlenecks and augmenting throughput efficiency. Extensive experiments were conducted with the collaborative simulation platform of CARLA and SUMO, and the experimental results show that the proposed PE-DDPG outperforms the baseline methods in terms of the convergence capacity of the training process, the smoothness of driving and the traffic efficiency with diverse traffic flow patterns and penetration rates of autonomous vehicles (AVs). Generally, the proposed PE-DDPG model could be employed for autonomous driving in complex scenarios with imperfect data.

show abstract

“…In [105], a driving policy based on the SAC algorithm combined with interval prediction and self-attention mechanism (IP-SAC) was presented to traverse unsignalized roundabouts. To maximize data usage and efficiency during the learning process, they used experience replay and randomly selected a batch of 256 items from the replay buffer.…”

Section: Roundaboutsmentioning

confidence: 99%

Autonomous Vehicle Decision-Making and Control in Complex and Unconventional Scenarios—A Review

2023

View full text Add to dashboard Cite

The development of autonomous vehicles (AVs) is becoming increasingly important as the need for reliable and safe transportation grows. However, in order to achieve level 5 autonomy, it is crucial that such AVs can navigate through complex and unconventional scenarios. It has been observed that currently deployed AVs, like human drivers, struggle the most in cases of adverse weather conditions, unsignalized intersections, crosswalks, roundabouts, and near-accident scenarios. This review paper provides a comprehensive overview of the various navigation methodologies used in handling these situations. The paper discusses both traditional planning methods such as graph-based approaches and emerging solutions including machine-learning based approaches and other advanced decision-making and control techniques. The benefits and drawbacks of previous studies in this area are discussed in detail and it is identified that the biggest shortcomings and challenges are benchmarking, ensuring interpretability, incorporating safety as well as road user interactions, and unrealistic simplifications such as the availability of accurate and perfect perception information. Some suggestions to tackle these challenges are also presented.

show abstract

Design of Unsignalized Roundabouts Driving Policy of Autonomous Vehicles Using Deep Reinforcement Learning

Cited by 9 publications

References 17 publications

A Comparative Study of Traffic Signal Control Based on Reinforcement Learning Algorithms

A Comparative Study of Traffic Signal Control Based on Reinforcement Learning Algorithms

Perception Enhanced Deep Deterministic Policy Gradient for Autonomous Driving in Complex Scenarios

Autonomous Vehicle Decision-Making and Control in Complex and Unconventional Scenarios—A Review

Contact Info

Product

Resources

About