Continuous residual reinforcement learning for traffic signal control optimization

Aslani, Mohammad Reza; Seipel, Stefan; Wiering, Marco A.

doi:10.1139/cjce-2017-0408

Cited by 17 publications

(11 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang L drew a conclusion that extensive simulation results for the designed Shanghai simulation scenarios indicate that most of the observed counts match quite well with the traffic simulation volumes and demonstrate the potential of MATSIM for large-scale dynamic transport simulation [18]. Aslani M developed adaptive traffic signal controllers based on continuous residual reinforcement learning (CRL-TSC) that was more stable, and the best setup of the CRL-TSC leads to saving average travel time by 15% in comparison to an optimized fixed-time controller [19].…”

Section: Reinforcement Learning Trafficmentioning

confidence: 69%

Evaluation and Application of Urban Traffic Signal Optimizing Control Strategy Based on Reinforcement Learning

Wang

Yang

Liu

et al. 2018

Journal of Advanced Transportation

View full text Add to dashboard Cite

Reinforcement learning method has a self-learning ability in complex multidimensional space because it does not need accurate mathematical model and due to the low requirement for prior knowledge of the environment. The single intersection, arterial lines, and regional road network of a group of multiple intersections are taken as the research object on the paper. Based on the three key parameters of cycle, arterial coordination offset, and green split, a set of hierarchical control algorithms based on reinforcement learning is constructed to optimize and improve the current signal timing scheme. However, the traffic signal optimization strategy based on reinforcement learning is suitable for complex traffic environments (high flows and multiple intersections), and the effects of which are better than the current optimization methods in the conditions of high flows in single intersections, arteries, and regional multi-intersection. In a word, the problem of insufficient traffic signal control capability is studied, and the hierarchical control algorithm based on reinforcement learning is applied to traffic signal control, so as to provide new ideas and methods for traffic signal control theory.

show abstract

Section: Reinforcement Learning Trafficmentioning

confidence: 69%

Evaluation and Application of Urban Traffic Signal Optimizing Control Strategy Based on Reinforcement Learning

Wang

Yang

Liu

et al. 2018

Journal of Advanced Transportation

View full text Add to dashboard Cite

show abstract

“…(Nuli and Mathew, 2013)), stability (e.g. (Aslani et al, 2018b)), speed of optimization (e.g. in Transfer Learning models (N. Xu et al, 2019)), state and/or action space manageability or generalizability (e.g.…”

Section: Methods' Contribution and Combinationmentioning

confidence: 99%

Reinforcement Learning in Urban Network Traffic Signal Control: A Systematic Literature Review

Noaeen¹,

Naik²,

Goodman³

et al. 2021

Preprint

View full text Add to dashboard Cite

Improvement of traffic signal control (TSC) efficiency has been found to lead to improved urban transportation and enhanced quality of life. Recently, the use of reinforcement learning (RL) in various areas of TSC has gained significant traction; thus, we conducted a systematic literature review as a systematic, comprehensive, and reproducible review to dissect all the existing research that applied RL in the network-level TSC (NTSC) domain. The review only targeted the network-level articles that tested the proposed methods in networks with two or more intersections. We used natural language processing to define the search strings and searched Google Scholar, Web of Science, IEEE Xplore, ACM Digital Library, Springer Link, and Science Direct databases. This review covers 160 peer-reviewed articles from 30 countries published from 1994 to March 2020. The goal of this study is to provide the research community with statistical and conceptual knowledge, summarize existence evidence, characterize RL applications in NTSC domains, explore all applied methods and major first events in the defined scope, and identify areas for further research based on the explored research problems in current research.

show abstract

“…Aslani et al [10] introduced the actor-critic method to solve the problem of the trade-off between exploration of the traffic environment and exploitation of the knowledge already obtained. Aslani et al [11] developed adaptive traffic signal controllers based on continuous residual reinforcement learning to improve their stability. Jeon et al [12] suggested a novel artificial intelligence that only uses video images of an intersection; the image-based RL model outperformed both the actual operation of fixed signals and a fully actuated operation.…”

Section: Review Of the Literature On Signal Coordinationmentioning

confidence: 99%

Optimizing the Junction-Tree-Based Reinforcement Learning Algorithm for Network-Wide Signal Coordination

Zhao

Shen

et al. 2020

Journal of Advanced Transportation

View full text Add to dashboard Cite

is study develops three measures to optimize the junction-tree-based reinforcement learning (RL) algorithm, which will be used for network-wide signal coordination. e first measure is to optimize the frequency of running the junction-tree algorithm (JTA) and the intersection status division. e second one is to optimize the JTA information transmission mode. e third one is to optimize the operation of a single intersection. A test network and three test groups are built to analyze the optimization effect. Group 1 is the control group, group 2 adopts the optimizations for the basic parameters and the information transmission mode, and group 3 adopts optimizations for the operation of a single intersection. Environments with different congestion levels are also tested. Results show that optimizations of the basic parameters and the information transmission mode can improve the system efficiency and the flexibility of the green light, and optimizing the operation of a single intersection can improve the efficiency of both the system and the individual intersection. By applying the proposed optimizations to the existing JTA-based RL algorithm, network-wide signal coordination can perform better.

show abstract

Continuous residual reinforcement learning for traffic signal control optimization

Cited by 17 publications

References 35 publications

Evaluation and Application of Urban Traffic Signal Optimizing Control Strategy Based on Reinforcement Learning

Evaluation and Application of Urban Traffic Signal Optimizing Control Strategy Based on Reinforcement Learning

Reinforcement Learning in Urban Network Traffic Signal Control: A Systematic Literature Review

Optimizing the Junction-Tree-Based Reinforcement Learning Algorithm for Network-Wide Signal Coordination

Contact Info

Product

Resources

About