Prosocial Norm Emergence in Multi-agent Systems

Mashayekhi, Mehdi; Ajmeri, Nirav; List, George; Singh, Munindar P.

doi:10.1145/3540202

Cited by 6 publications

(7 citation statements)

References 72 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The resultant social-interaction equilibrium achieved is termed convention emergence or norm emergence. These equilibrium goals may vary depending on the objective of the desired agent behavior, i.e., finding the optimal policy, a Nash equilibrium, or reaching Pareto equilibrium (Mashayekhi et al, 2022). The system has reached norm convergence when a significant agent adopts similar norms and the system achieves stability.…”

Section: Norm Emergencementioning

confidence: 99%

Norm Augmented Reinforcement Learning Agents With Synthesized Normative Rules

Kadir,

Selamat,

Krejcar

2024

Journal of Cases on Information Technology

View full text Add to dashboard Cite

The dynamic deontic (DD) is a norm synthesis framework that extracts normative rules from reinforcement learning (RL), however it was not designed to be applied in agent coordination. This study proposes a norm augmented reinforcement learning framework (NARLF) that extends said model to include a norm deliberation mechanism for learned norms re-imputation for norm biased decision-making RL agents. This study aims to test the effects of synthesized norms applied on-line and off-line on agent learning performance. The framework consists of the DD framework extended with a pre-processing and deliberation component to allow re-imputation of normative rules. A deliberation model, the Norm Augmented Q-Table (NAugQT), is proposed to map normative rules into RL agents via q-values weight updates. Results show that the framework is able to map and improve RL agent's performance but only when synthesized off-line edited absolute norm salience value norms are used. This shows limitations when unstable salience norms are applied. Improvement in norm extraction and pre-processing are required.

show abstract

Section: Norm Emergencementioning

confidence: 99%

Norm Augmented Reinforcement Learning Agents With Synthesized Normative Rules

Kadir,

Selamat,

Krejcar

2024

Journal of Cases on Information Technology

View full text Add to dashboard Cite

show abstract

“…In MAS, norms refer to mechanisms that regulate the agent's behavior and specify the responsibilities agents have towards each other 9 . Norms enable agents to demand certain behaviors from others, promoting cooperation and ensuring effective coordination in autonomous but interdependent MASs 40,41 .…”

Section: Norms and Normative Systemsmentioning

confidence: 99%

A framework for creating systems capable of adapting and dealing with norms based on software agents

Viana

Alencar

Cowan

et al. 2023

Preprint

View full text Add to dashboard Cite

The software for distributed intelligent systems, also known as Multi-agent Systems (MASs), must possess the capability to comply with environmental constraints, such as norms, and exhibit self-adaptive mechanisms to autonomously modify their behavior in response to contextual changes and handle adverse situations independently. However, there is a lack of comprehensive understanding of the concepts of self-adaptation and normative systems, as well as their behavior and interactions, which is often not adequately supported by software tools. This research introduces an extension to the Framework for Normative Agent Java Simulation (JSAN), enabling implement normative adaptive agents based on an architecture capable of dealing with norms through self-adaptation. The extended framework provides support for the key properties, namely self-adaptation, norms, and normative reasoning. To validate the efficacy of this approach, a virtual marketplace study case is presented, showcasing the agents' ability to adapt to norms while facilitating user transactions and product purchases.

show abstract

“…In MARL, there are several schemas to introduce different sociality modes [27,28]. In this paper, we propose two parametrised functions to vary the sociality settings for the predator-prey game.…”

Section: Sociality Regimes Parametrisation For Predators and Prey In Mpementioning

confidence: 99%

When Being Selfish Prevails: The Impact of Sociality Regimes on Heterogeneous Cooperative-Competitive Multi-agent Reinforcement Learning

Zhao

Hernández-Orallo

2022

Preprint

View full text Add to dashboard Cite

Work in multi-agent reinforcement learning (MARL) tends to compare algorithms choosing some particular sociality regime for sharing (or not) the rewards, assuming this choice is optimal. For instance, the popular multi-agent deep deterministic policy gradient algorithm (MADDPG), when evaluated in a predator and prey game, used a different regime when acting as a predator than when acting as a prey. In this paper, we question what kind of sociality regimes (selfish, egalitarian, or altruistic) should be used in multi-agent algorithms. In the past, choices of the sociality regime have been done only for homogeneous teams, and without normalising the effect of the sociality regimes. Here we introduce a normalisation of the weights of the sociality regimes so that the sum of all rewards is independent of the sociality regime. With this, we extend the analysis to heterogeneous teams in cooperative-competitive MARL situations since team composition is very relevant to the sociality choice. We find that the selfish regime is advantageous for both prey and predator teams, and for both homogeneous and heterogeneous teams. The takeaway message is that, unlike common practice in MARL, any study of homogeneous and heterogeneous cooperative-competitive multi-agent reinforcement learning teams should also take into account the sociality regimes before making conclusions on the preference of any algorithm.

show abstract

Prosocial Norm Emergence in Multi-agent Systems

Cited by 6 publications

References 72 publications

Norm Augmented Reinforcement Learning Agents With Synthesized Normative Rules

Norm Augmented Reinforcement Learning Agents With Synthesized Normative Rules

A framework for creating systems capable of adapting and dealing with norms based on software agents

When Being Selfish Prevails: The Impact of Sociality Regimes on Heterogeneous Cooperative-Competitive Multi-agent Reinforcement Learning

Contact Info

Product

Resources

About