A traffic light control method based on multi-agent deep reinforcement learning algorithm

Abstract

Intelligent traffic light control (ITLC) algorithms are very efficient for relieving traffic congestion. Recently, many decentralized multi-agent traffic light control algorithms are proposed. These researches mainly focus on improving reinforcement learning method and coordination method. But, as all the agents need to communicate while coordinating with each other, the communication details should be improved as well. To guarantee communication effectiveness, two aspect should be considered. Firstly, a traffic condition description method need to be designed. By using this method, traffic condition can be described simply and clearly. Secondly, synchronization should be considered. As different intersections have different cycle lengths and message sending event happens at the end of each traffic signal cycle, every agent will receive messages of other agents at different time. So it is hard for an agent to decide which message is the latest one and the most valuable. Apart from communication details, reinforcement learning algorithm used for traffic signal timing should also be improved. In the traditional reinforcement learning based ITLC algorithms, either queue length of congested cars or waiting time of these cars is considered while calculating reward value. But, both of them are very important. So a new reward calculation method is needed. To solve all these problems, in this paper, a new ITLC algorithm is proposed. To improve communication efficiency, this algorithm adopts a new message sending and processing method. Besides, to measure traffic congestion in a more reasonable way, a new reward calculation method is proposed and used. This method takes both waiting time and queue length into consideration.

Enhancing Urban Intersection Efficiency: Visible Light Communication and Learning-Based Control for Traffic Signal Optimization and Vehicle Management

“…The effect of the adjacent intersections with the same structure is the so-called adjacent symmetric homogenous reward [35,36]. Such a cooperative mechanism helps to balance the traffic flow between intersections and learn better in both intersections with one agent in each intersection.…”

A Dynamic Traffic Light Control Algorithm to Mitigate Traffic Congestion in Metropolitan Areas

“…From a survey of the literature mentioned above, researchers mention two primary issues, namely, vehicle backlog and fixed signal timing. Particularly, [ 11 ] considered the traditional signal time, and the traditional DQN-based intelligent traffic light control (ITLC) method was used to determine the new incentive amount. At times of high demand, the current fixed signal timing flow model cannot adequately control traffic flow.…”

Enhancing the Robustness of Traffic Signal Control with StageLight: A Multiscale Learning Approach

“…Their work explored three variants of consistent dual definition for state and reward based on the queue length, number of vehicles, and waiting time. In another study, Liu and Li [4] proposed a multiagent-based algorithm that enhanced interagency communication and gauged traffic congestion in a more reasonable way. They introduced a new reward that incorporated both the waiting time and queue length, aiming for a comprehensive congestion assessment.…”