Design of reinforcement learning for perimeter control using network transmission model based macroscopic traffic simulation

Abstract

Perimeter control is an emerging alternative for traffic signal control, which regulates the traffic flows on the periphery of a road network. Some model-based approaches have been suggested earlier for the optimization of perimeter control based on macroscopic fundamental diagrams (MFDs). However, there are several limitations when considering their application to a large-scale urban area because the model-based approaches may not be scalable to multiple regions and inappropriate for handling various effects caused by the shape change of MFDs. Therefore, we propose a model-free and data-driven approach that combines reinforcement learning (RL) with the macroscopic traffic simulation based on the recently developed network transmission model. First, we design four perimeter control models with different macroscopic traffic variables and parametrizations. Then, we validate the proposed models by evaluating their performances with the test demand scenarios at different levels. The validation results show that the model containing travel demand information adapts to a new demand scenario better than the model containing only density-related factors.

Methodology to Increase Flexibility in Inter-Region Flow Control for Urban Traffic

“…The problems become even more complex if the controller is centralized with many urban regions. Therefore, a few recent studies have made efforts to develop machine-learning-based approaches (28,29) or distributed control approaches to reduce the optimization complexity (21,22,30), but coordinating the individual decisions of multiple control agents is still a difficult task for improving the global performance.…”