How Well Do Reinforcement Learning Approaches Cope With Disruptions? The Case of Traffic Signal Control

Korecki, Marcin; Dailisan, Damian N.; Helbing, Dirk

doi:10.1109/access.2023.3266644

Cited by 3 publications

(2 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It provides a real-world example of a complex system which proves highly challenging for traditional deep learning as well as meta-learning algorithms. The need for lifelong adaptation is also clear in the traffic system, where unpredictable events such as accidents, jams, and roadwork can occur daily (the effects of such disruptions on learning algorithms in the context of traffic has been investigated in [30]). In order for a traffic signal control algorithm to be actually useful, the learning algorithm would need to enable continuous learning and be highly adaptable.…”

Section: Discussionmentioning

confidence: 99%

Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control

Korecki

2023

Entropy

Self Cite

View full text Add to dashboard Cite

We studied the ability of deep reinforcement learning and self-organizing approaches to adapt to dynamic complex systems, using the applied example of traffic signal control in a simulated urban environment. We highlight the general limitations of deep learning for control in complex systems, even when employing state-of-the-art meta-learning methods, and contrast it with self-organization-based methods. Accordingly, we argue that complex systems are a good and challenging study environment for developing and improving meta-learning approaches. At the same time, we point to the importance of baselines to which meta-learning methods can be compared and present a self-organizing analytic traffic signal control that outperforms state-of-the-art meta-learning in some scenarios. We also show that meta-learning methods outperform classical learning methods in our simulated environment (around 1.5–2× improvement, in most scenarios). Our conclusions are that, in order to develop effective meta-learning methods that are able to adapt to a variety of conditions, it is necessary to test them in demanding, complex settings (such as, for example, urban traffic control) and compare them against established methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep Reinforcement Meta-Learning and Self-Organization in Complex Systems: Applications to Traffic Signal Control

Korecki

2023

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…The aim of this is to increase the agent's adaptability to various conditions. Nevertheless, some concerns have been raised regarding the resilience of RL solutions to disruptions and their performance compared to previously existing, non-learning algorithms (Korecki et al, 2023b).…”

Section: Traffic Signal Control In Urban Trafficmentioning

confidence: 99%