Deep Reinforcement Learning Based Left-Turn Connected and Automated Vehicle Control at Signalized Intersection in Vehicle-to-Infrastructure Environment

Abstract

In order to solve the problem of vehicle delay caused by stops at signalized intersections, a micro-control method of a left-turning connected and automated vehicle (CAV) based on an improved deep deterministic policy gradient (DDPG) is designed in this paper. In this paper, the micro-control of the whole process of a left-turn vehicle approaching, entering, and leaving a signalized intersection is considered. In addition, in order to solve the problems of low sampling efficiency and overestimation of the critic network of the DDPG algorithm, a positive and negative reward experience replay buffer sampling mechanism and multi-critic network structure are adopted in the DDPG algorithm in this paper. Finally, the effectiveness of the signal control method, six DDPG-based methods (DDPG, PNRERB-1C-DDPG, PNRERB-3C-DDPG, PNRERB-5C-DDPG, PNRERB-5CNG-DDPG, and PNRERB-7C-DDPG), and four DQN-based methods (DQN, Dueling DQN, Double DQN, and Prioritized Replay DQN) are verified under 0.2, 0.5, and 0.7 saturation degrees of left-turning vehicles at a signalized intersection within a VISSIM simulation environment. The results show that the proposed deep reinforcement learning method can get a number of stops benefits ranging from 5% to 94%, stop time benefits ranging from 1% to 99%, and delay benefits ranging from −17% to 93%, respectively compared with the traditional signal control method.

Real-Time Adaptive Traffic Signal Control in a Connected and Automated Vehicle Environment: Optimisation of Signal Planning with Reinforcement Learning under Vehicle Speed Guidance

“…Reinforcement learning-based adaptive traffic signal control changes traffic signals based on the feedback from the traffic demand, which can be hypothetical dynamic [ 41 , 42 , 43 , 44 ] or based on real-world data [ 45 , 46 ]. The existing literature on using the reinforcement learning approach can be categorised into two groups: networks consisting of CVs [ 44 , 47 , 48 ] and non-CVs environments [ 45 , 46 , 49 ]. Moreover, two general classifications (i.e., vehicle positions and queue length) are available for state representation.…”

“…Furthermore, all of the papers relevant to this topic have used simulation platforms in order to obtain their desired results. SUMO [ 43 , 44 , 49 , 50 ], VISSIM [ 46 , 47 ], AIMSUN [ 45 ], and PARAMICS [ 30 ] are the most common software packages in which the combination of traffic simulation and RL can be executed appropriately. Finally, it is worth mentioning that all the previous papers took the signal controller as an agent for their RL algorithm except [ 47 ], which used connected vehicles as its agents.…”

An Efficiency Enhancing Methodology for Multiple Autonomous Vehicles in an Urban Network Adopting Deep Reinforcement Learning

“…For example, Tan et al [26] used the DRL for large-scale adaptive traffic signal control (ATSC). Chen et al [27] applied DRL to left turn CAVs at a signalized intersection. Kim and Jeong [28] applied DRL to control multiple signalized intersections.…”

Low-Frequency Signal Sampling Method Implemented in a PLC Controller Dedicated to Applications in the Monitoring of Selected Electrical Devices

“…Navigation and traffic jam warning systems also show great potential for the use of low-frequency methods [22] and low-frequency sampling methods can be used in monitoring weather parameters [23].…”