Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

Abstract

The transportation demand is rapidly growing in metropolises, resulting in chronic traffic congestions in dense downtown areas. Adaptive traffic signal control as the principle part of intelligent transportation systems has a primary role to effectively reduce traffic congestion by making a real-time adaptation in response to the changing traffic network dynamics. Reinforcement learning (RL) is an effective approach in machine learning that has been applied for designing adaptive traffic signal controllers. One of the most efficient and robust type of RL algorithms are continuous state actor-critic algorithms that have the advantage of fast learning and the ability to generalize to new and unseen traffic conditions. These algorithms are utilized in this paper to design adaptive traffic signal controllers called actor-critic adaptive traffic signal controllers (A-CATs controllers). The contribution of the present work rests on the integration of three threads: (a) showing performance comparisons of both discrete and continuous A-CATs controllers in a traffic network with recurring congestion (24-hour traffic demand) in the upper downtown core of Tehran city, (b) analyzing the effects of different traffic disruptions including opportunistic pedestrians crossing, parking lane, non-recurring congestion, and different levels of sensor noise on the performance of A-CATS controllers, and (c) comparing the performance of different function approximators (tile coding and radial basis function) on the learning of A-CATs controllers. To this end, first an agent-based traffic simulation of the study area is carried out. Then six different scenarios are conducted to find the best A-CATs controller that is robust enough against different traffic disruptions. We observe that the A-CATs controller based on radial basis function networks (RBF (5)) outperforms others. This controller is benchmarked against controllers of discrete state Q-learning, Bayesian Q-learning, fixed time and actuated controllers; and the results reveal that it consistently outperforms them.

Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control

“…Thus kernel method was applied to extract nonlinear features from low-dimensional states [28]. Kernel method was also applied in LR actorcritic recently, under realistically simulated traffic environments [14]. Alternatively, natural actor-critic was applied to improve the fitting accuracy of LR in ATSC [29].…”

“…1) Action definition: There are several standard action definitions, such as phase switch [38], phase duration [14], and phase itself [26]. We follow the last definition and simply define each local action as a possible phase, or red-green combinations of traffic lights at that intersection.…”

Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

“…Also, in this short paper we focus our analysis on a single intersection as a first step for understanding deep RL robustness. As in previous efforts for robustness analysis [8], [19], network and coordination can have significant impacts, but is left for future research. 1) Network: A simple 4-leg intersection as represented in Fig.…”

Pengembangan Model Simulasi Sistem Dinamik Untuk Meningkatkan Efisiensi Sistem Operasional Transportasi

“…Tujuan dari sistem transportasi adalah untuk mencapai proses transportasi penumpang dan barang secara optimum dalam ruang dan waktu tertentu, dengan mempertimbangkan faktor keamanan, kelancaran dan kenyamanan, serta waktu dan biaya. Menurut Jeihani et,al., dan Bifulco et al, Beberapa faktor/variabel terbesar yang mempengaruhi arus lalu lintas dalam sistem transportasi yaitu kemacetan, kemacetan dapat disebabkan oleh adanya gangguan yang tidak terduga seperti kecelakaan, pengerjaan jalan dan kerusakan jalan serta kondisi cuaca yang buruk [2], dapat juga disebabkan adanya hambatan lalu lintas, biasanya terkait dengan adanya bus, truk atau kendaraan lambat lainnya yang berada di jalur lalu lintas perkotaan [3].…”