Handling uncertainty in self-adaptive systems: an ontology-based reinforcement learning model

Ghanadbashi, Saeedeh; Safavifar, Zahra; Taebi, Farshad; Golpayegani, Fatemeh

doi:10.1007/s40860-022-00198-x

Cited by 6 publications

(1 citation statement)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our model can be extended to a large-scale network. Ghanadbashi et al (2023) introduces a model called OnCertain to improve decision-making in selfadaptive systems that interact with each other in dynamic environments. The proposed system can handle uncertainty caused by unpredictable and rare events while having limited information about the environment.…”

Section: Robustness In Traffic Signal Controlmentioning

confidence: 99%

Improving the Generalizability and Robustness of Large-Scale Traffic Signal Control

Shi,

Devailly,

Larocque

et al. 2024

IEEE Open J. Intell. Transp. Syst.

View full text Add to dashboard Cite

A number of deep reinforcement-learning (RL) approaches propose to control traffic signals.Compared to traditional approaches, RL approaches can learn from higher-dimensionality input road and vehicle sensors and better adapt to varying traffic conditions resulting in reduced travel times (in simulation). However, these RL methods require training from massive traffic sensor data. To offset this relative inefficiency, some recent RL methods have the ability to first learn from small-scale networks and then generalize to unseen city-scale networks without additional retraining (zero-shot transfer). In this work, we study the robustness of such methods along two axes. First, sensor failures and GPS occlusions create missing-data challenges and we show that recent methods remain brittle in the face of these missing data. Second, we provide a more systematic study of the generalization ability of RL methods to new networks with different traffic regimes. Again, we identify the limitations of recent approaches. We then propose using a combination of distributional and vanilla reinforcement learning through a policy ensemble. Building upon the state-of-the-art previous model which uses a decentralized approach for large-scale traffic signal control with graph convolutional networks (GCNs), we first learn models using a distributional reinforcement learning (DisRL) approach. In particular, we use implicit quantile networks (IQN) to model the state-action return distribution with quantile regression. For traffic signal control problems, an ensemble of standard RL and DisRL yields superior performance across different scenarios, including different levels of missing sensor data and traffic flow patterns. Furthermore, the learning scheme of the resulting model can improve zero-shot transferability to different road network structures, including both synthetic networks and real-world networks (e.g., Luxembourg, Manhattan). We conduct extensive experiments to compare our approach to multi-agent reinforcement learning and traditional transportation approaches. Results show that the proposed method improves robustness and generalizability in the face of missing data, varying road networks, and traffic flows.

show abstract

Section: Robustness In Traffic Signal Controlmentioning

confidence: 99%