Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning

Li, Zhenning; Yu, Hao; Zhang, Guohui; Dong, Shangjia; Xu, Cheng‐Zhong

doi:10.1016/j.trc.2021.103059

Cited by 83 publications

(50 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…One-hundred forty-four fully fledged research articles were finally selected based on the following inclusion criteria: most relevant, most cited, and most recent. For traffic state forecasting, in terms of performances, the GAN-based methods and also hybrid approaches showed better performance on state-of-the-art datasets, i.e., PeMS (Li et al [154], Zhang et al [95]). For intersection signal control, DRL-and DQN-based approaches showed much better efficiency and robustness (Wang et al [155], Bouktif et al [9]) relative to other baselines.…”

Section: Discussionmentioning

confidence: 99%

“…Luo et al [153] combined DL and RL by utilizing the MDP and CNN, which reduced the queue length by 42.5% relative to DQN. Considering knowledge sharing among the agents, Li et al [154] proposed the Knowledge-Sharing Deep Deterministic Policy Gradient (KS-DDPG) algorithm, which showed significant efficiency in controlling large-scale networks and coping with fluctuations in traffic flow. The inability of DRL algorithms to meet the demands of coordination among the agents inspired Wang et al [155] to propose a Cooperative Group-Based Multi-agent reinforcement learning-ATSC (CGB-MATSC) framework that demonstrated a significant reduction of average waiting time by 42.08% relative to FT. Kekuda et al [156] proposed an n-step State, Action, Reward, State, and Action (SARSA) algorithm to increase the implementability in low-cost real-time systems and compared it with LQF; it showed a 5.5% reduction of the average queue length.…”

Section: Neural Network Controllermentioning

confidence: 99%

“…Multi-agent deep-reinforcement-learningbased methods possess the provision to cope with this. For example, the knowledge-sharing deep deterministic policy gradient algorithm showed an average reduction of the queue length and intersection delay by 28.9% and 35.1%, respectively, relative to the MaxPressure (MP) method (Li et al [154]).…”

Section: Comparative Analysismentioning

confidence: 99%

See 2 more Smart Citations

Urban Safety: An Image-Processing and Deep-Learning-Based Intelligent Traffic Management and Control System

Reza

Oliveira

Machado

et al. 2021

Sensors

View full text Add to dashboard Cite

With the rapid growth and development of cities, Intelligent Traffic Management and Control (ITMC) is becoming a fundamental component to address the challenges of modern urban traffic management, where a wide range of daily problems need to be addressed in a prompt and expedited manner. Issues such as unpredictable traffic dynamics, resource constraints, and abnormal events pose difficulties to city managers. ITMC aims to increase the efficiency of traffic management by minimizing the odds of traffic problems, by providing real-time traffic state forecasts to better schedule the intersection signal controls. Reliable implementations of ITMC improve the safety of inhabitants and the quality of life, leading to economic growth. In recent years, researchers have proposed different solutions to address specific problems concerning traffic management, ranging from image-processing and deep-learning techniques to forecasting the traffic state and deriving policies to control intersection signals. This review article studies the primary public datasets helpful in developing models to address the identified problems, complemented with a deep analysis of the works related to traffic state forecast and intersection-signal-control models. Our analysis found that deep-learning-based approaches for short-term traffic state forecast and multi-intersection signal control showed reasonable results, but lacked robustness for unusual scenarios, particularly during oversaturated situations, which can be resolved by explicitly addressing these cases, potentially leading to significant improvements of the systems overall. However, there is arguably a long path until these models can be used safely and effectively in real-world scenarios.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Neural Network Controllermentioning

confidence: 99%

Section: Comparative Analysismentioning

confidence: 99%

See 1 more Smart Citation

Urban Safety: An Image-Processing and Deep-Learning-Based Intelligent Traffic Management and Control System

Reza

Oliveira

Machado

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Yao et al proposed a dynamic predictive control framework for traffic signal control in a cross-sectional vehicle infrastructure integration environment [11]. Li et al proposed a multi-agent reinforcement learning method to achieve optimal traffic control by enhancing the cooperation between traffic signals [12]. By introducing the knowledge-sharing enabled communication protocol, each agent can access to the collective representation of the traffic environment collected by all agents.…”

Section: Related Workmentioning

confidence: 99%

A Distributed Urban Traffic Congestion Prevention Mechanism for Mixed Flow of Human-Driven and Autonomous Electric Vehicles

Huang¹,

Hu²,

Ho³

et al. 2021

IJCIS

View full text Add to dashboard Cite

Traffic congestion in urban areas has become a critical problem that municipal governments cannot overlook. Meanwhile, mixed traffic systems containing both autonomous and human-driven electric vehicles ramp up the challenge for traffic management in urban areas. Although numerous researchers have proposed traffic control heuristics to alleviate traffic congestion problems in the recent literature, scant research has addressed the joint problems of route and charging strategies for electric vehicles along with urban traffic congestion prevention. Accordingly, this work tackles the complex task of traffic management in urban areas during peak periods by using practical congestion prevention strategies that consider the characteristics of mixed traffic flows and the charging demands of electric vehicle users. Notably, we apply support vector regressions to compute the charging time at each charging point and the traverse time of an electric vehicle at each road segment/intersection, based on historical traffic data. The simulation results reveal that the proposed algorithms are feasible because they can avoid possible occurrences of traffic congestion during rush hours and provide the routes and charging options that are chosen by electric vehicle users.

show abstract

“…In [26], the authors investigated a multi-agent algorithm based on Q-learning, taking into account the traffic state at neighboring intersections. In [27], the authors proposed using a knowledge exchange protocol between agents to increase the level of cooperation between agents and achieve an optimal traffic light control strategy. A double Q-learning algorithm for improving the stability of control policy was investigated in [28].…”

Section: Related Workmentioning

confidence: 99%

Traffic Signal Control: a Double Q-learning Approach

Agafonov¹,

Myasnikov²

2021

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

Currently, the use of information and communication technologies for solving economic, social, transportation, and other problems in the urban environment is usually considered within the "smart city" concept. Optimal traffic management and, in particular, traffic signal control is one of the key components of smart cities. In this paper, we investigate the reinforcement learning approach, namely, the double Q-learning approach, to solve the traffic signal control problem. Both the initial data on the connected vehicles distribution and the aggregated characteristics of traffic flows are used to describe the state of the reinforcement learning agent. Experimental studies of the proposed model were carried out on synthetic and real data using the CityFlow microscopic traffic simulator.

show abstract

Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning

Cited by 83 publications

References 38 publications

Urban Safety: An Image-Processing and Deep-Learning-Based Intelligent Traffic Management and Control System

Urban Safety: An Image-Processing and Deep-Learning-Based Intelligent Traffic Management and Control System

A Distributed Urban Traffic Congestion Prevention Mechanism for Mixed Flow of Human-Driven and Autonomous Electric Vehicles

Traffic Signal Control: a Double Q-learning Approach

Contact Info

Product

Resources

About