A Sustainable Ecosystem through Emergent Cooperation in Multi-Agent Reinforcement Learning

Ritz, Fabian; Ratke, Daniel; Phan, Thomy; Belzner, Lenz; Linnhoff‐Popien, Claudia

doi:10.1162/isal_a_00399

Cited by 3 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ideally, this adaptation should be autonomous, i.e., the change of parameters in data exchange should be decided without even knowing the application or the agents' backgrounds. This objective fits well in the field of emergent communications that studies which circumstances lead to communication as an instrumental strategy when multiple learning agents are rewarded for completing specific tasks [14]. Knowledge distillation -With advancements in federated learning techniques, knowledge can be aggregated as individually trained models are integrated.…”

Section: B Emergent Knowledge Accumulationmentioning

confidence: 79%

Effective Communications for 6G: Challenges and Opportunities

Soyak¹,

Erçetin²

2022

Preprint

View full text Add to dashboard Cite

This article studies effective communication, one of the three forms identified by Weaver and Shannon, as an enabler for the upcoming 6G use cases. The envisioned tactile, holographic, and multi-sensory communications require bandwidths in the order of terabits per second and latencies in the order of microseconds for an immersive experience. We argue that a theoretical framework for transporting information tailored to endusers' goals is necessary to support such applications. Different from the recently emerging discussions focusing on the meaning of exchanged messages, we focus on using these messages to take actions in the desired way. We highlight the essential characteristics of distributed knowledge accumulation as a facilitator for this upcoming paradigm, and discuss the challenges of making effective communications a reality and the potential opportunities for future research to address these challenges. In a reallife use case, we showcase the potential reduction in the number of bits transferred owing to the transferred accumulated knowledge.

show abstract

Section: B Emergent Knowledge Accumulationmentioning

confidence: 79%

Effective Communications for 6G: Challenges and Opportunities

Soyak¹,

Erçetin²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…( [44,26,6,49]. Different studies have been conducted on various complex SSDs, where interesting phenomena like group hunting, attacking and dodging, or flocking have been observed [23,31,15,36]. Independent MARL like naive learning has been widely used in most studies to model agents with individual rationality [44,13].…”

Section: Policy Gradient Reinforcement Learningmentioning

confidence: 99%

“…Multi-agent reinforcement learning (MARL) has become popular to model individually rational agents in SDs and SSDs to examine emergent behavior [6,23,31,15,36]. The goal of each agent is defined by an individual reward function.…”

Section: Introductionmentioning

confidence: 99%

Emergent Cooperation from Mutual Acknowledgment Exchange in Multi-Agent Reinforcement Learning

Phan

Sommer

Ritz

et al. 2022

Preprint

View full text Add to dashboard Cite

Peer incentivization (PI) is a recent approach, where all agents learn to reward or to penalize each other in a distributed fashion which often leads to emergent cooperation. Current PI mechanisms implicitly assume a flawless communication channel in order to exchange rewards. These rewards are directly integrated into the learning process without any chance to respond with feedback. Furthermore, most PI approaches rely on global information which limits scalability and applicability to real-world scenarios, where only local information is accessible. In this paper, we propose Mutual Acknowledgment Token Exchange (MATE), a PI approach defined by a two-phase communication protocol to mutually exchange acknowledgment tokens to shape individual rewards. Each agent evaluates the monotonic improvement of its individual situation in order to accept or reject acknowledgment requests from other agents. MATE is completely decentralized and only requires local communication and information. We evaluate MATE in three social dilemma domains. Our results show that MATE is able to achieve and maintain significantly higher levels of cooperation than previous PI approaches. In addition, we evaluate the robustness of MATE in more realistic scenarios, where agents can defect from the protocol and where communication failures can occur. We also evaluate the sensitivity of MATE w.r.t. the choice of token values.

show abstract

Emergent cooperation from mutual acknowledgment exchange in multi-agent reinforcement learning

Phan,

Sommer,

Ritz

et al. 2024

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

Peer incentivization (PI) is a recent approach where all agents learn to reward or penalize each other in a distributed fashion, which often leads to emergent cooperation. Current PI mechanisms implicitly assume a flawless communication channel in order to exchange rewards. These rewards are directly incorporated into the learning process without any chance to respond with feedback. Furthermore, most PI approaches rely on global information, which limits scalability and applicability to real-world scenarios where only local information is accessible. In this paper, we propose Mutual Acknowledgment Token Exchange (MATE), a PI approach defined by a two-phase communication protocol to exchange acknowledgment tokens as incentives to shape individual rewards mutually. All agents condition their token transmissions on the locally estimated quality of their own situations based on environmental rewards and received tokens. MATE is completely decentralized and only requires local communication and information. We evaluate MATE in three social dilemma domains. Our results show that MATE is able to achieve and maintain significantly higher levels of cooperation than previous PI approaches. In addition, we evaluate the robustness of MATE in more realistic scenarios, where agents can deviate from the protocol and communication failures can occur. We also evaluate the sensitivity of MATE w.r.t. the choice of token values.

show abstract

A Sustainable Ecosystem through Emergent Cooperation in Multi-Agent Reinforcement Learning

Cited by 3 publications

References 14 publications

Effective Communications for 6G: Challenges and Opportunities

Effective Communications for 6G: Challenges and Opportunities

Emergent Cooperation from Mutual Acknowledgment Exchange in Multi-Agent Reinforcement Learning

Emergent cooperation from mutual acknowledgment exchange in multi-agent reinforcement learning

Contact Info

Product

Resources

About