Explainable and Adaptable Augmentation in Knowledge Attention Network for Multi-Agent Deep Reinforcement Learning Systems

Ho, Joshua W. K.; Wang, Chien‐Min

doi:10.1109/aike48582.2020.00031

Cited by 12 publications

(9 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, like any other centralized system, team learning requests full swarm connectivity and depends on a leading node which is a single point of failure to the system. Concurrent learning is a distributed network of agents that handles individual learning processes in parallel [23] [24]. Hence, agents need to co-adapt and find optimal cooperative behaviors from individual and global reinforcements.…”

Section: Fig 1 Simplified Map Of Sample-based Learning Methods [17]mentioning

confidence: 99%

“…The developed algorithm is tested and validated in the hunter-prey pursuit environment against distributed Q-Learning TD method. The pursuit game has been a reference to MARL problem since almost its existence [25] [24], involving cooperation from a group of learning agents that, following a pursuer role, try to capture one or multiple evaders. As a result of the tabular setting, this paper contemplates a grid world with only two intelligent hunters and a random behavior prey, although future research includes transitioning to the function approximation sphere.…”

Section: Fig 1 Simplified Map Of Sample-based Learning Methods [17]mentioning

confidence: 99%

“…The hunter-prey pursuit problem has been used as a benchmark in MARL for years. Continuous state spaces [33] [31], grid domains [24] [25] and obstacles [23] are some of the modified environment features that authors have considered in their respective research. This section contains a description of environment key aspects and fundamental rules that define the experimentation setup.…”

Section: Problem Formulationmentioning

confidence: 99%

See 2 more Smart Citations

Swarm Intelligence in Cooperative Environments: Introducing the N-Step Dynamic Tree Search Algorithm

Longa

Tsourdos

2022

AIAA SCITECH 2022 Forum

View full text Add to dashboard Cite

Uncertainty and partial or unknown information about environment dynamics have led reward-based methods to play a key role in the Single-Agent and Multi-Agent Learning problem. Tree-based planning approaches such as Monte Carlo Tree Search algorithm have been a striking success in single-agent domains where a perfect simulator model is available, e.g., Go and chess strategic board games. This paper presents a decentralized tree-based planning scheme, that combines forward planning with direct reinforcement learning temporal-difference updates applied to the multi-agent setting. Forward planning requires an engine model which is learned from experience and represented via function approximation. Evaluation and validation are carried out in the Hunter-Prey Pursuit cooperative environment and performance is compared with state-of-the-art RL techniques. N-Step Dynamic Tree Search (NSDTS) pretends to adapt the most successful single-agent learning methods to the multi-agent boundaries in a decentralized system structure, dealing with scalability issues and exponential growth of computational resources suffered by centralized systems. NSDTS demonstrates to be a remarkable advance compared to the conventional Q-Learning temporal-difference method.

show abstract

Section: Fig 1 Simplified Map Of Sample-based Learning Methods [17]mentioning

confidence: 99%

Section: Fig 1 Simplified Map Of Sample-based Learning Methods [17]mentioning

confidence: 99%

Section: Problem Formulationmentioning

confidence: 99%

See 1 more Smart Citation

Swarm Intelligence in Cooperative Environments: Introducing the N-Step Dynamic Tree Search Algorithm

Longa

Tsourdos

2022

AIAA SCITECH 2022 Forum

View full text Add to dashboard Cite

show abstract

“…Furthermore, like any other centralized system, team learning requests full swarm connectivity and is reliant on a critical leading node. On the other hand, concurrent learning [5] utilizes a distributed network of agents to handle multiple learning processes simultaneously [23] [24]. Thus, co-adaption through reinforcements is required to achieve optimal cooperation.…”

Section: Introductionmentioning

confidence: 99%

Swarm Intelligence in Cooperative Environments: N-Step Dynamic Tree Search Algorithm Extended Analysis

Longa

Tsourdos

2022

2022 American Control Conference (ACC)

View full text Add to dashboard Cite

Reinforcement learning tree-based planning methods have been gaining popularity in the last few years due to their success in single-agent domains, where a perfect simulator model is available, e.g., Go and chess strategic board games. This paper pretends to extend tree search algorithms to the multi-agent setting in a decentralized structure, dealing with scalability issues and exponential growth of computational resources. The N-Step Dynamic Tree Search combines forward planning and direct temporal-difference updates, outperforming markedly state-of-the-art algorithms such as Q-Learning and SARSA. Future state transitions and rewards are predicted with a model built and learned from real interactions between agents and the environment. As an extension of previous work, this paper analyses the developed algorithm in the Hunter-Pursuit cooperative game against intelligent evaders. The N-Step Dynamic Tree Search aims to adapt the most successful singleagent learning methods to the multi-agent boundaries and demonstrates to be a remarkable advance compared to conventional temporal-difference techniques.

show abstract

“…The explainability problem is relevant for multi-agent networks and it has been receiving increased attention [16]- [18]. While cooperation is beneficial over such networks, leading to more stable performance, it nevertheless makes interpretability of the results more challenging especially when the underlying topology is unknown to the observer.…”

Section: Introductionmentioning

confidence: 99%

Explainability and Graph Learning from Social Interactions

Shumovskaia¹,

Ntemos²,

Vlaski³

et al. 2022

Preprint

View full text Add to dashboard Cite

Social learning algorithms provide models for the formation of opinions over social networks resulting from local reasoning and peer-to-peer exchanges. Interactions occur over an underlying graph topology, which describes the flow of information among the agents. In this work, we propose a technique that addresses questions of explainability and interpretability when the graph is hidden. Given observations of the evolution of the belief over time, we aim to infer the underlying graph topology, discover pairwise influences between the agents, and identify significant trajectories in the network. The proposed framework is online in nature and can adapt dynamically to changes in the graph topology or the true hypothesis.

show abstract

Explainable and Adaptable Augmentation in Knowledge Attention Network for Multi-Agent Deep Reinforcement Learning Systems

Cited by 12 publications

References 10 publications

Swarm Intelligence in Cooperative Environments: Introducing the N-Step Dynamic Tree Search Algorithm

Swarm Intelligence in Cooperative Environments: Introducing the N-Step Dynamic Tree Search Algorithm

Swarm Intelligence in Cooperative Environments: N-Step Dynamic Tree Search Algorithm Extended Analysis

Explainability and Graph Learning from Social Interactions

Contact Info

Product

Resources

About