Traffic Light Control by Multiagent Reinforcement Learning Systems

Abstract

Traffic light control is one of the main means of controlling road traffic. Improving traffic control is important because it can lead to higher traffic throughput and reduced congestion. This chapter describes multiagent reinforcement learning techniques for automatic optimization of traffic light controllers. Such techniques are attractive because they can automatically discover efficient control strategies for complex tasks, such as traffic control, for which it is hard or impossible to compute optimal solutions directly and hard to develop hand-coded solutions. First the general multi-agent reinforcement learning framework is described that is used to control traffic lights in this work. In this framework, multiple local controllers (agents) are each responsible for the optimization of traffic lights around a single traffic junction, making use of locally perceived traffic state information (sensed cars on the road), a learned probabilistic model of car behavior, and a learned value function which indicates how traffic light decisions affect long-term utility, in terms of the average waiting time of cars. Next, three extensions are described which improve upon the basic framework in various ways: agents (traffic junction controllers) taking into account congestion information from neighboring agents; handling partial observability of traffic states; and coordinating the behavior of multiple agents by coordination graphs and the max-plus algorithm.

From Goods to Traffic: First Steps Toward an Auction-Based Traffic Signal Controller

“…Although simple, similar networks have been used in other tra c experiments [3,4]. A single Grid City block measures 200 meters.…”

“…Tra c lights are probably the most prevalent means of controlling tra c. Other methods include stop signs and roundabouts. Although many tra c lights rely on simple fixed protocols, they are none-the-less a vital component of tra c management [4]. More advanced adaptive Urban Tra c Controllers (UTC), such as RHODES [17], OPAC [11] and SCOOT 1 , have been developed in an e↵ort to improve the performance of tra c lights [22,18].…”

Multi-intersections traffic signal intelligent control using collaborative q-learning algorithm

“…Step 4：Agent calculate reward value t r based on equation (5)and update learning rate t a based on equation (6).…”

“…Since traffic control is fundamentally a problem of sequential decision making, and at the same time is a task that is too complex for straightforward computation of optimal solutions or effective hand-coded solutions, it is perhaps best suited to the framework ofMarkov Decision Processes (MDPs) and reinforcement learning (RL) or approximate dynamic programming (ADP), in which an agent learns from trial and error via interaction with its environment [5].…”

An Intersection-Centric Auction-Based Traffic Signal Control Framework

“…Poorly designed phase plans may lead to additional delays, traffic jams and even accidents. Although many traffic signals rely on simple fixed protocols, they are nonetheless a vital component of traffic management [1]. Historically, finding the best signal timings involved using mathematical models of traffic behaviour to determine ideal settings [22].…”