Distributed Signal Control of Arterial Corridors Using Multi-Agent Deep Reinforcement Learning

Zhang, Weibin; Yan, Chen; Li, Xiaofeng; Fang, Liangliang; Wu, Yao; Li, Jun

doi:10.1109/tits.2022.3216203

Cited by 10 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section evaluates the performance of OAS Deep Q-Learning is compared with those of two categories of the baseline methods: traditional methods (including MaxPressure [ 49 ] and FixedTime [ 50 ]) and the Multi-Agent Deep Reinforcement Learning (MARL) [ 51 ], Meta Variationally Intrinsic Motivated Reinforcement Learning (MetaVIMRL) [ 52 ], and Cooperative Multi-Agent Deep Q-Network (CMDQN) [ 53 ] based on the training results. For fair comparisons, the parameters and simulation conditions of the traditional methods and the DQL methods [ 51 , 52 , 53 ] are the same. Table 9 presents the parameters of OAS Deep Q-Learning with reinforcement learning network as mentioned in this paper.…”

Section: Resultsmentioning

confidence: 99%

“…To test the efficacy of the model proposed in this paper, the traffic flow data at the intersections 1-3 are presented in Tables 6-8. This section evaluates the performance of OAS Deep Q-Learning is compared with those of two categories of the baseline methods: traditional methods (including MaxPressure [49] and FixedTime [50]) and the Multi-Agent Deep Reinforcement Learning (MARL) [51], Meta Variationally Intrinsic Motivated Reinforcement Learning (MetaVIMRL) [52], and Cooperative Multi-Agent Deep Q-Network (CMDQN) [53] based on the training results. For fair comparisons, the parameters and simulation conditions of the traditional methods and the DQL methods [51][52][53] are the same.…”

Section: Collection Of Data On Arterial Intersectionsmentioning

confidence: 99%

See 1 more Smart Citation

OAS Deep Q-Learning-Based Fast and Smooth Control Method for Traffic Signal Transition in Urban Arterial Tidal Lanes

Dong,

Xie,

et al. 2024

Sensors

View full text Add to dashboard Cite

To address traffic flow fluctuations caused by changes in traffic signal control schemes on tidal lanes and maintain smooth traffic operations, this paper proposes a method for controlling traffic signal transitions on tidal lanes. Firstly, the proposed method includes designing an intersection overlap phase scheme based on the traffic flow conflict matrix in the tidal lane scenario and a fast and smooth transition method for key intersections based on the flow ratio. The aim of the control is to equalize average queue lengths and minimize average vehicle delays for different flow directions at the intersection. This study also analyses various tidal lane scenarios based on the different opening states of the tidal lanes at related intersections. The transitions of phase offsets are emphasized after a comprehensive analysis of transition time and smoothing characteristics. In addition, this paper proposes a coordinated method for tidal lanes to optimize the phase offset at arterial intersections for smooth and rapid transitions. The method uses Deep Q-Learning, a reinforcement learning algorithm for optimal action selection (OSA), to develop an adaptive traffic signal transition control and enhance its efficiency. Finally, a simulation experiment using a traffic control interface is presented to validate the proposed approach. This study shows that this method leads to smoother and faster traffic signal transitions across different tidal lane scenarios compared to the conventional method. Implementing this solution can benefit intersection groups by reducing traffic delays, improving traffic efficiency, and decreasing air pollution caused by congestion.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Collection Of Data On Arterial Intersectionsmentioning

confidence: 99%

OAS Deep Q-Learning-Based Fast and Smooth Control Method for Traffic Signal Transition in Urban Arterial Tidal Lanes

Dong,

Xie,

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…When primary vehicles are close to an intersection, coordinated phases can turn green in time without constraints; however, there is a large safety risk owing to deviations from the real-world traffic environment. The other type [1,22,23] considers these constraints and can achieve similar vehicle progression through explicit signal coordination. The weights of the neural networks embed the maintenance of vehicle progression and improvement of system performance.…”

Section: Rl Methods For Signal Coordinationmentioning

confidence: 99%

“…To achieve this, the primary vehicles should be delayed as less as possible by the downstream signals. In real-world arterials, the phase sequence at an intersection is typically fixed over time for safety reasons [1]. This means that every signal must display green in a cyclical and fixed sequence, and that the primary vehicles arriving at the downstream intersection cyclically cannot be served in a timely manner.…”

Section: Introductionmentioning

confidence: 99%

Explicit coordinated signal control using soft actor–critic for cycle length determination

Zhang,

Xu,

Pan

et al. 2024

IET Intelligent Trans Sys

View full text Add to dashboard Cite

Explicit signal coordination carries prior knowledge of traffic engineering and is widely accepted for global implementation. With the recent popularity of reinforcement learning, numerous researchers have turned to implicit signal coordination. However, these methods inevitably require learning coordination from scratch. To maximize the use of prior knowledge, this study proposes an explicit coordinated signal control (ECSC) method using a soft actor–critic for cycle length determination. This method can fundamentally solve the challenges encountered by traditional methods in determining the cycle length. Soft actor–critic was selected among various reinforcement learning methods. A single agent was administered to the arterials. An action is defined as the selection of a cycle length from among the candidates. The state is represented as a feature vector, including the cycle length and features of each leg at every intersection. The reward is defined as departures that indirectly minimize system vehicle delays. Simulation results indicate that ECSC significantly outperforms the baseline methods, as evident in system vehicle delay across nearly all demand scenarios and throughput in high demand scenarios. The ECSC revitalizes explicit signal coordination and introduces new perspectives on the application of reinforcement learning methods in signal coordination.

show abstract

An Approach to Multi-agent Deep Q-Network Optimization of Signal Control in Multi-intersection Road Environments to Enhance Urban Traffic Flow

Ergün

2024

Communications in Computer and Information Science

View full text Add to dashboard Cite

Distributed Signal Control of Arterial Corridors Using Multi-Agent Deep Reinforcement Learning

Cited by 10 publications

References 29 publications

OAS Deep Q-Learning-Based Fast and Smooth Control Method for Traffic Signal Transition in Urban Arterial Tidal Lanes

OAS Deep Q-Learning-Based Fast and Smooth Control Method for Traffic Signal Transition in Urban Arterial Tidal Lanes

Explicit coordinated signal control using soft actor–critic for cycle length determination

An Approach to Multi-agent Deep Q-Network Optimization of Signal Control in Multi-intersection Road Environments to Enhance Urban Traffic Flow

Contact Info

Product

Resources

About