Hierarchical control of traffic signals using Q-learning with tile coding

Abdoos, Monireh; Mozayani, Nasser; Bazzan, Ana L. C.

doi:10.1007/s10489-013-0455-3

Cited by 57 publications

(34 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results demonstrated that the policy gradient actor-critic method consistently outperforms the continuous Q-learning algorithm. Abdoos et al (2014) employed the integration of discrete RL and continuous RL to control 9 traffic signals on a 3 × 3 junction grid. There are two types of agents: bottom level agents and top level agents.…”

Section: Related Workmentioning

confidence: 99%

Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

Aslani

Mesgari

Wiering

2017

Transportation Research Part C: Emerging Technologies

176

View full text Add to dashboard Cite

The transportation demand is rapidly growing in metropolises, resulting in chronic traffic congestions in dense downtown areas. Adaptive traffic signal control as the principle part of intelligent transportation systems has a primary role to effectively reduce traffic congestion by making a real-time adaptation in response to the changing traffic network dynamics. Reinforcement learning (RL) is an effective approach in machine learning that has been applied for designing adaptive traffic signal controllers. One of the most efficient and robust type of RL algorithms are continuous state actor-critic algorithms that have the advantage of fast learning and the ability to generalize to new and unseen traffic conditions. These algorithms are utilized in this paper to design adaptive traffic signal controllers called actor-critic adaptive traffic signal controllers (A-CATs controllers). The contribution of the present work rests on the integration of three threads: (a) showing performance comparisons of both discrete and continuous A-CATs controllers in a traffic network with recurring congestion (24-hour traffic demand) in the upper downtown core of Tehran city, (b) analyzing the effects of different traffic disruptions including opportunistic pedestrians crossing, parking lane, non-recurring congestion, and different levels of sensor noise on the performance of A-CATS controllers, and (c) comparing the performance of different function approximators (tile coding and radial basis function) on the learning of A-CATs controllers. To this end, first an agent-based traffic simulation of the study area is carried out. Then six different scenarios are conducted to find the best A-CATs controller that is robust enough against different traffic disruptions. We observe that the A-CATs controller based on radial basis function networks (RBF (5)) outperforms others. This controller is benchmarked against controllers of discrete state Q-learning, Bayesian Q-learning, fixed time and actuated controllers; and the results reveal that it consistently outperforms them.

show abstract

Section: Related Workmentioning

confidence: 99%

Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

Aslani

Mesgari

Wiering

2017

Transportation Research Part C: Emerging Technologies

176

View full text Add to dashboard Cite

show abstract

“…In [13] the authors propose using a set of Q-learning iterations to approach the optimal solution of load balancing. They also mentioned several methods to control the traffic lights and intersections using different techniques of the artificial intelligence, such as fuzzy rules, predefined rule-based systems, and centralized methods.…”

Section: Automobile Applicationsmentioning

confidence: 99%

Multiagent Systems in Automotive Applications

Campos-Rodríguez¹,

Gonzalez-Jimenez²,

FranciscoCervantes‐Alvarez³

et al. 2017

Multi-Agent Systems

View full text Add to dashboard Cite

The multiagent systems have proved to be a useful tool in the design of solutions to problems of distributed nature. In a distributed system, it is possible that the data, the control actions or even both, be distributed. The concept of agent is a suitable notion for capturing situations where the global knowledge about the status of a system is complex or even impossible to acquire in a single entity. In automotive applications, there exist a great number of scenarios of distributed nature, such as the traffic coordination, routes load balancing problems, traffic negotiation among the infrastructure and cars, to mention a few. Even more, the autonomous driving features of the new generation of cars will require the new methods of car to car communication, car to infrastructure negotiation, and even infrastructure to infrastructure communication. This chapter proposes the application of multiagent system techniques to some problems in the automotive field.

show abstract

“…This solution concept is robust to many forms of noise, and does not require expensive recalculations or a complex teamwork protocol. Additionally, we do not impose an explicitly defined organization and communication structure on top of the system as in some work [57], including work with hierarchies [58,59], and instead only rely on the teamwide performance being broadcast to every team member (the team game assumption).…”

Section: Multiagent Systemsmentioning

confidence: 99%

Autonomous Multiagent Space Exploration with High-Level Human Feedback

Colby

Yliniemi

Tumer

2016

Journal of Aerospace Information Systems

View full text Add to dashboard Cite

Robotic space-exploration missions have always pushed the limits of science and technology, and will continue to do so by their very nature. Such missions are particularly challenging, as they operate in environments with high uncertainty, light-time delays, and high mission costs. Artificial-intelligence-based multiagent systems can alleviate these concerns by 1) creating autonomous multirobot teams that can function in uncertain environments, 2) navigating and operating without time-sensitive commands from Earth-bound scientists, and 3) spreading the mission cost across multiple platforms that will eliminate the danger of total mission loss in the case of a malfunctioning robot. In this work, a novel human in-the-loop cooperative coevolutionary algorithm is presented to train a multirobot system exploring an unknown environment. Autonomous robots learn to make low-level control decisions to maximize scientific data acquisition, whereas human scientists on Earth learn the changing mission profiles and provide highlevel objectives to the robots. Results demonstrate that the algorithm reduces the number of robots needed for a particular performance level tenfold compared to traditional cooperative coevolutionary algorithms for configurations of 10 or more rovers, resulting in significantly lower mission costs. Further, the trained multirobot system is extremely robust to noise, and 10% sensor and actuator noise (with and without sensor bias) has no statistically significant impact on system performance. Finally, the system is extremely robust to robot failures; for any percentage of robot failures between 10 and 90%, the percentage loss in performance is less than the percentage of failed robots.

show abstract

Hierarchical control of traffic signals using Q-learning with tile coding

Cited by 57 publications

References 17 publications

Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

Multiagent Systems in Automotive Applications

Autonomous Multiagent Space Exploration with High-Level Human Feedback

Contact Info

Product

Resources

About