Soilse: A decentralized approach to optimization of fluctuating urban traffic using Reinforcement Learning

Salkham, As'ad; Cahill, Vinny

doi:10.1109/itsc.2010.5625145

Cited by 15 publications

(14 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AI-based autonomous traffic systems use computer vision, data analytics and machine learning techniques for their operation. Improvement in computing power for RL has catapulted its use in traffic systems, and, RL-based traffic signal controllers are being designed [52,56,61]. Non-stationary RL traffic controllers are proposed by References [52,61].…”

Section: Transportation and Traffic Systemsmentioning

confidence: 99%

“…Improvement in computing power for RL has catapulted its use in traffic systems, and, RL-based traffic signal controllers are being designed [52,56,61]. Non-stationary RL traffic controllers are proposed by References [52,61].…”

Section: Transportation and Traffic Systemsmentioning

confidence: 99%

“…Soilse [61] is a RL-based intelligent urban traffic signal controller (UTC) tailored to fluctuating vehicular traffic patterns. It is phase-based, wherein a phase is a set of traffic lanes on which vehicles are allowed to proceed at any given time.…”

Section: Transportation and Traffic Systemsmentioning

confidence: 99%

See 2 more Smart Citations

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

Padakandla

2021

ACM Comput. Surv.

View full text Add to dashboard Cite

Reinforcement learning (RL) algorithms find applications in inventory control, recommender systems, vehicular traffic management, cloud computing, and robotics. The real-world complications arising in these domains makes them difficult to solve with the basic assumptions underlying classical RL algorithms. RL agents in these applications often need to react and adapt to changing operating conditions. A significant part of research on single-agent RL techniques focuses on developing algorithms when the underlying assumption of stationary environment model is relaxed. This article provides a survey of RL methods developed for handling dynamically varying environment models. The goal of methods not limited by the stationarity assumption is to help autonomous agents adapt to varying operating conditions. This is possible either by minimizing the rewards lost during learning by RL agent or by finding a suitable policy for the RL agent that leads to efficient operation of the underlying system. A representative collection of these algorithms is discussed in detail in this work along with their categorization and their relative merits and demerits. Additionally, we also review works that are tailored to application domains. Finally, we discuss future enhancements for this field.

show abstract

Section: Transportation and Traffic Systemsmentioning

confidence: 99%

Section: Transportation and Traffic Systemsmentioning

confidence: 99%

See 1 more Smart Citation

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

Padakandla

2021

ACM Comput. Surv.

View full text Add to dashboard Cite

show abstract

“…State of the Art Such problems have also been approached from the environment perspective in traffic (Salkham and Cahill, 2010), where the SOILSE method is proposed. Fluctuations in environment behaviour are continuously monitored using CUSUM, a moving average filter.…”

Section: Environment-induced Non-stationaritymentioning

confidence: 99%

Prediction-Based Multi-Agent Reinforcement Learning in Inherently Non-Stationary Environments

Marinescu

Dusparić

Clarke

2017

ACM Trans. Auton. Adapt. Syst.

View full text Add to dashboard Cite

Multi-agent reinforcement learning (MARL) is a widely researched technique for decentralised control in complex large-scale autonomous systems. Such systems often operate in environments that are continuously evolving and where agents’ actions are non-deterministic, so called inherently non-stationary environments. When there are inconsistent results for agents acting on such an environment, learning and adapting is challenging. In this article, we propose P-MARL, an approach that integrates prediction and pattern change detection abilities into MARL and thus minimises the effect of non-stationarity in the environment. The environment is modelled as a time-series, with future estimates provided using prediction techniques. Learning is based on the predicted environment behaviour, with agents employing this knowledge to improve their performance in realtime. We illustrate P-MARL’s performance in a real-world smart grid scenario, where the environment is heavily influenced by non-stationary power demand patterns from residential consumers. We evaluate P-MARL in three different situations, where agents’ action decisions are independent, simultaneous, and sequential. Results show that all methods outperform traditional MARL, with sequential P-MARL achieving best results.

show abstract

“…Although the use of several approaches to investigate traffic management issues, the majority of these propositions are based on the use of reinforcement and Qlearning techniques [7][8][9][10][11][12][13][14][15][16]. Among those works, the timearrival estimation technique introduced in [7] proposed a prediction engine system that built its visions and decisions based on the context behaviors of drivers and vehicles.…”

Section: Related Workmentioning

confidence: 99%

Context-aware multi-modal traffic management in ITS: A Q-learning based algorithm

Said

Soua

Abd-Elrahman

et al. 2015

2015 International Wireless Communications and Mobile Computing Conference (IWCMC)

View full text Add to dashboard Cite

Multi-modal traffic management in IntelligentTransportation Systems (ITS) aims to provide a more efficient traffic regulation to passengers and reduce congestion and obstruction in the roads. In spite of the outstanding progress made in this research filed; traffic management still a very challenging problem regarding the multiple factors that have to be taken into account in any proposed solution. To tackle this problem, this paper introduces a collaborative model based context awareness multi-modal traffic management aiming at providing an efficient way to manage the traffic inside a transportation station. In this model (Multi-Layers Stations: the stations that have different intersections for different means of transport), the traffic management is based on a Q-learning technique that takes into account the context awareness parameters to provide more potent decisions. The learning technique offers the opportunity to the system (transportation station) to adapt dynamically its decision (choice of the best transportation mean) based on feedbacks provided by the passengers traveling from that specified station and thus optimize their journey through the transportation network. The efficacy of our proposed technique is validated through extensive simulations for different layers of transport means like metros, trains, and buses. Our proposal holds for any ITS system decisions provided the availability of real-time traces about the passengers passing by any station.

show abstract

Soilse: A decentralized approach to optimization of fluctuating urban traffic using Reinforcement Learning

Cited by 15 publications

References 30 publications

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

Prediction-Based Multi-Agent Reinforcement Learning in Inherently Non-Stationary Environments

Context-aware multi-modal traffic management in ITS: A Q-learning based algorithm

Contact Info

Product

Resources

About