An Experimental Review of Reinforcement Learning Algorithms for Adaptive Traffic Signal Control

Mannion, Patrick; Duggan, Jim; Howley, Enda

doi:10.1007/978-3-319-25808-9_4

Cited by 216 publications

(159 citation statements)

References 35 publications

Supporting

Mentioning

143

Contrasting

Unclassified

Order By: Relevance

“…As other works have already proved the efficacy of RL-TSC approaches in test scenarios with multiple intersections (see e.g. 2,8 ), we have decided to focus on using single junction test cases to clearly illustrate the benefit of our PRL approach without adding unnecessary additional complexity. Table 1.…”

Section: Experimental Designmentioning

confidence: 98%

“…Traffic control problems make very attractive testbeds for emerging RL approaches, and present a number of non-trivial challenges such as developing strategies for coordination and information sharing between individual agents. For a comprehensive review of the usage of learning agents in Traffic Signal Control, we refer the interested reader to a review paper published by Mannion et al 8 .…”

Section: Parallel Reinforcement Learning For Traffic Signal Controlmentioning

confidence: 99%

“…Reinforcement Learning has been shown to be a promising approach when applied to urban traffic signal control (e.g. 2,8,9 ), and offers many benefits; RL agents can learn online to continuously improve their performance, as well as adapting readily to changes in traffic demand. Traffic control problems make very attractive testbeds for emerging RL approaches, and present a number of non-trivial challenges such as developing strategies for coordination and information sharing between individual agents.…”

Section: Parallel Reinforcement Learning For Traffic Signal Controlmentioning

confidence: 99%

See 2 more Smart Citations

Parallel Reinforcement Learning for Traffic Signal Control

Mannion

Duggan

Howley

2015

Procedia Computer Science

Self Cite

View full text Add to dashboard Cite

Developing Adaptive Traffic Signal Control strategies for efficient urban traffic management is a challenging problem, which is not easily solved. Reinforcement Learning (RL) has been shown to be a promising approach when applied to traffic signal control (TSC) problems. When using RL agents for TSC, difficulties may arise with respect to convergence times and performance. This is especially pronounced on complex intersections with many different phases, due to the increased size of the state action space. Parallel Learning is an emerging technique in RL literature, which allows several learning agents to pool their experiences while learning concurrently on the same problem. Here we present an extension to a leading published work on RL for TSC, which leverages the benefits of Parallel Learning to increase exploration and reduce delay times and queue lengths.

show abstract

Section: Experimental Designmentioning

confidence: 98%

Section: Parallel Reinforcement Learning For Traffic Signal Controlmentioning

confidence: 99%

Section: Parallel Reinforcement Learning For Traffic Signal Controlmentioning

confidence: 99%

See 1 more Smart Citation

Parallel Reinforcement Learning for Traffic Signal Control

Mannion

Duggan

Howley

2015

Procedia Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Agents in a cooperative MAS are designed to work together to achieve a system-level goal [11]. Numerous complex, real world systems have been successfully optimised using the MAS framework, including air traffic control [12], traffic signal control [2], data routing in networks [13], electricity generator scheduling [14,15], RoboCup soccer [16] and water resource management [17].…”

Section: Multi-agent Reinforcement Learningmentioning

confidence: 99%

“…Single-objective approaches seek to find a single solution to a problem, whereas in reality a system may have multiple conflicting objectives that could be optimised. Examples of multi-objective problems include stock market forecasting [1], traffic signal control [2] and load balancing in smart grids [3].…”

Section: Introductionmentioning

confidence: 99%

Policy invariance under reward transformations for multi-objective reinforcement learning

et al. 2017

Self Cite

View full text Add to dashboard Cite

Reinforcement Learning (RL) is a powerful and well-studied Machine Learning paradigm, where an agent learns to improve its performance in an environment by maximising a reward signal. In multi-objective Reinforcement Learning (MORL) the reward signal is a vector, where each component represents the performance on a different objective. Reward shaping is a wellestablished family of techniques that have been successfully used to improve the performance and learning speed of RL agents in single-objective problems. The basic premise of reward shaping is to add an additional shaping reward to the reward naturally received from the environment, to incorporate domain knowledge and guide an agent's exploration. Potential-Based Reward Shaping (PBRS) is a specific form of reward shaping that offers additional guarantees. In this paper, we extend the theoretical guarantees of PBRS to MORL problems. Specifically, we provide theoretical proof that PBRS does not alter the true Pareto front in both single-and multi-agent MORL. We also contribute the first published empirical studies of the effect of PBRS in single-and multi-agent MORL problems.

show abstract

Emerging Memristive Artificial Synapses and Neurons for Energy‐Efficient Neuromorphic Computing

2020

View full text Add to dashboard Cite

as the thermal budget has increased rapidly in recent years, which could surpass the future revenue generated worldwide by the semiconductor industry. [4,5] Consequently, electronic miniaturization limits the sustainable growth of computing technology. The von Neumann design is a conventional computing architecture, which separates the processor and memory unit. This architecture performs a computational task through sequential procedures and has functioned a mainstay of modern computing since 1945. [6] In fact, the architecture is significantly beneficial to both hardware and software developers because each component can be improved and readily extended into a united electronic system, even without a comprehensive understanding of all the components. However, the von Neumann bottleneck generates substantial power consumption and latency during the computing operation. [2,7] This limitation results from data transmission between two functional units (processor and memory), particularly when the memory is accessed through a bus with restricted bandwidth. [2,7] Moreover, according to the International Data Corporation (IDC), global data will rapidly increase to 175 ZB (1.75 × 10 23 B) by 2025. [8] Consequently, the von Neumann bottleneck will be more detrimental under tremendous workload because it may be produced by the daily operation of the design. Recently, there has been increasing demand for intellectual computers that can efficiently process substantial global data, thereby resulting in the development of a wide range of softwareor hardware-based artificial neural networks (ANNs) aimed at achieving the computing ability of the human brain. [2,9] Softwarebased ANNs have recently exhibited remarkable capabilities, such as image recognition, [10,11] natural language processing, [12,13] and performing specific tasks, [14] some beyond the human level. However, since existing ANNs are built on conventional computing architecture, that is, the von Neumann architecture, the learning parameters stored in memory are iteratively introduced to the processor to perform a task. The bottleneck issue arising from the movement of large datasets would eventually reduce the energy and time efficiency when operating the ANN software. [2,7] To alleviate this problem, several advanced software-and hardware-based ANN approaches have been suggested. [2,7,15-17] Advanced algorithms, such as network pruning, quantization, Huffman coding, and knowledge distillation, have been Memristors have recently attracted significant interest due to their applicability as promising building blocks of neuromorphic computing and electronic systems. The dynamic reconfiguration of memristors, which is based on the history of applied electrical stimuli, can mimic both essential analog synaptic and neuronal functionalities. These can be utilized as the node and terminal devices in an artificial neural network. Consequently, the ability to understand, control, and utilize fundamental switching principles and various types of device architectures of the...

show abstract

An Experimental Review of Reinforcement Learning Algorithms for Adaptive Traffic Signal Control

Cited by 216 publications

References 35 publications

Parallel Reinforcement Learning for Traffic Signal Control

Parallel Reinforcement Learning for Traffic Signal Control

Policy invariance under reward transformations for multi-objective reinforcement learning

Emerging Memristive Artificial Synapses and Neurons for Energy‐Efficient Neuromorphic Computing

Contact Info

Product

Resources

About