Real-Time Coordinated Signal Control Through Use of Agents with Online Reinforcement Learning

Abstract

A multiagent architecture for real-time coordinated signal control in an urban traffic network is introduced. The multiagent architecture consists of three hierarchical layers of controller agents: intersection, zone, and regional controllers. Each controller agent is implemented by applying artificial intelligence concepts, namely, fuzzy logic, neural network, and evolutionary algorithm. From the fuzzy rule base, each individual controller agent recommends an appropriate signal policy at the end of each signal phase. These policies are later processed in a policy repository before being selected and implemented into the traffic network. To handle the changing dynamics of the complex traffic processes within the network, an online reinforcement learning module is used to update the knowledge base and inference rules of the agents. This concept of a multiagent system with online reinforcement learning was implemented in a network consisting of 25 signalized intersections in a microscopic traffic simulator. Initial test results showed that the multiagent system improved average delay and total vehicle stoppage time, compared with the effects of fixed-time traffic signal control.

Applications of Artificial Intelligence in Transport: An Overview

“…The first system is to control traffic signal and the second one is to predict future traffic congestion. While, reference [72] demonstrated the feasibility of using NNs to control the traffic by proposing a multi-layer NNs system evaluated in three intersections networks [72].…”

Artificial Intelligence in Transportation: Information for Application

“…Software agents are designed to function autonomously and control tasks. For example, Choy et al (2003) and Manikonda et al (2001) use software agents to optimize traffic signal timing. Bonabeau (2002a) suggests that ABM is appropriate • When there is potential for emergent phenomena-that is, individual behavior is nonlinear and changes over time, and fluctuations are more important than averages;…”

Test Bed for Multiagent Control Systems in Road Traffic Management

“…The agents formulate actions to be performed and propose them to a 'coordinator', who makes a final decision in case of conflicting plans. The authors of [4] introduce middle-level zone controller agents and highest-level region controller agents to coordinate the actions of the intersection controller agents that are present at the lowest level. In both of these approaches there is no communication between agents at the same hierarchical level.…”

“…This delegation of tasks into various subtasks is an active field of research in the world of distributed artificial intelligence. Others have argued that multiagent systems can aid in the distribution of the problem and facilitate the coordination of the activities of the traffic control instruments when needed (1)(2)(3)(4)(5)(6)(7)(8).…”