Reinforcement learning (RL) is a promising datadriven approach for adaptive traffic signal control (ATSC) in complex urban traffic networks, and deep neural networks further enhance its learning power. However, centralized RL is infeasible for large-scale ATSC due to the extremely high dimension of the joint action space. Multi-agent RL (MARL) overcomes the scalability issue by distributing the global control to each local RL agent, but it introduces new challenges: now the environment becomes partially observable from the viewpoint of each local agent due to limited communication among agents. Most existing studies in MARL focus on designing efficient communication and coordination among traditional Q-learning agents. This paper presents, for the first time, a fully scalable and decentralized MARL algorithm for the state-of-the-art deep RL agent: advantage actor critic (A2C), within the context of ATSC. In particular, two methods are proposed to stabilize the learning procedure, by improving the observability and reducing the learning difficulty of each local agent. The proposed multi-agent A2C is compared against independent A2C and independent Q-learning algorithms, in both a large synthetic traffic grid and a large real-world traffic network of Monaco city, under simulated peak-hour traffic dynamics. Results demonstrate its optimality, robustness, and sample efficiency over other state-ofthe-art decentralized MARL algorithms.
Abstract-Different research communities varying from telecommunication to traffic engineering are working on problems related to vehicular traffic congestion, intelligent transportation systems, and mobility patterns using information collected from a variety of sensors. To test the solutions, the first step is to use a vehicular traffic simulator with an appropriate scenario in order to reproduce realistic mobility patterns. Many mobility simulators are available, and the choice is usually done based on the size and type of simulation required, but a common problem is to find a realistic traffic scenario. In order to evaluate and compare new communication protocols for vehicular networks, it is necessary to use a wireless network simulator in combination with a vehicular traffic simulator. This additional step introduces further requirements for the scenario. The aim of this work is to provide a scenario able to meet all the common requirements in terms of size, realism and duration, in order to have a common basis for the evaluations. In the interest of building a realistic scenario, we decided to start from a real city with a standard topology common in mid-size European cities, and real information concerning traffic demands and mobility patterns. In this paper we show the process used to build the Luxembourg SUMO Traffic (LuST) Scenario, and present a summary of its characteristics together with an overview of its possible use cases.
Cooperative Intelligent Transportation Systems (C-ITS) are a viable solution when it comes to the optimization of the ever-growing population moving in the cities. C-ITS studies have to deal with telecommunications issues and location errors due to the urban environment, while keeping into account realistic mobility patterns. A detailed and state of the art scenario is complex to generate and validate. There is a trade-off between precision and scalability. Additionally, precise information may be problematic to obtain or use due to privacy issues. There are some general-purpose freely-available scenarios, but none of them provides a 3D environment with intermodal traffic. Nonetheless, the 3D environment is a requirement to have reliable C-ITS simulations in a realistic setting, and the importance of intermodal mobility cannot be overlooked in planning the future of smart cities. The Monaco SUMO Traffic (MoST) Scenario aims to provide a state of the art 3D playground with various kind of vehicles, vulnerable road users and public transports to test C-ITS solutions. This paper presents the data requirements, characteristics, possible use cases, and finally, the limitations of MoST Scenario.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.