A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles

Abstract

The advent of connected vehicle (CV) technology offers new possibilities for a revolution in future transportation systems. With the availability of real-time traffic data from CVs, it is possible to more effectively optimize traffic signals to reduce congestion, increase fuel efficiency, and enhance road safety. The success of CV-based signal control depends on an accurate and computationally efficient model that accounts for the stochastic and nonlinear nature of the traffic flow. Without the necessity of prior knowledge of the traffic system’s model architecture, reinforcement learning (RL) is a promising tool to acquire the control policy through observing the transition of the traffic states. In this paper, we propose a novel data-driven traffic signal control method that leverages the latest in deep learning and reinforcement learning techniques. By incorporating a compressed representation of the traffic states, the proposed method overcomes the limitations of the existing methods in defining the action space to include more practical and flexible signal phases. The simulation results demonstrate the convergence and robust performance of the proposed method against several existing benchmark methods in terms of average vehicle speeds, queue length, wait time, and traffic density.

A two‐stage robust optimal traffic signal control with reversible lane for isolated intersections

“…In recent years, many scholars have focused on the method of traffic signal control under uncertainty. Various models handling uncertainty have been introduced, such as mean-standard deviation model (MSD), the conditional value-at-risk model and the min-max model [22], discretization model [23], constrained min-max model [24], dynamics model [25], two-stage stochastic planning method [26], offline scenario-based framework [27], and multiagent reinforcement learning [28,29]. These models also show their effectiveness and robustness in the problem of traffic signal control.…”

CVDMARL: A Communication-Enhanced Value Decomposition Multi-Agent Reinforcement Learning Traffic Signal Control Method

“…In the reinforcement learning (RL)-based control framework, the traffic signal control system no longer relies on heuristic assumptions and equations, but learns to optimize the signal control strategy through continuous trial and error through real-time interactions with a road network. Therefore, compared to traditional traffic control methods, RL signal control methods can usually achieve better control effects [8][9][10]. Early RL-based models solved traffic signal control problems by querying qtables that recorded the traffic state, actions, and rewards [11,12].…”

Simulation in the GPenSIM Environment of the Movement of Vehicles in the City Based on Their License Plate Numbers

“…Combining crossroads throughout the city into a single system allows for even better management of traffic signal control. Systems based on artificial intelligence are already being used and tested in China [24,25]. More and more, algorithms are not only able to identify objects (like cars and people), but they can also discern and interpret vehicle license plates.…”