SA-IGA: a multiagent reinforcement learning method towards socially optimal outcomes

Zhang, Chengwei; Li, Xiaohong; Hao, Jianye; Chen, Siqi; Tuyls, Karl; Xue, Wanli; Feng, Zhiyong

doi:10.1007/s10458-019-09411-3

Cited by 19 publications

(11 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second layer contributes details of how agents' characteristics determine the degree of empathy and make it possible to carry out an adaptive design for the decision algorithm. Given p ij = f (x), works in [24], [36] take into account the factors of changing empathy, such as social comparison and companion impression, and define f (•) in an incremental form. There are also some works that give the explicit function directly.…”

Section: Artificial Empathymentioning

confidence: 99%

“…It mainly discusses how to make group decisions to maximize social welfare in the known network of empathy weights. The work in [24] also aimed to maximize social welfare, but the difference is that the proposed algorithm is designed in a distributed learning form. As a whole, most articles do not study how the empathy model maps individual characteristics to the degree of empathy and remain focus on how to get rid of the economic dilemma and maximize social welfare under the action of altruism.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Artificial Empathy: A New Perspective for Analyzing and Designing Multi-Agent Systems

Chen

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Learning from biological mechanisms is an essential method of devising interaction rules among agents. Inspired by neuropsychology, we introduce empathy, a vital ability of higher animals' affective systems, to assist us in analyzing and designing multi-agent systems. In this paper, we abstract the process of empathy as an optimization problem and establish a reasonable model of empathy by optimizing the corresponding free energy. Variable temperatures on the integrated utility associated with empathy provide agents with several different modes, including collectivity, equality, oligopoly, and monopoly. Therefore, we can change the agent's mode artificially according to the task requirement and examine agents' evolution from the perspective of continuous changes on temperature. Then we present a bandit algorithm called Empathy-based Interactive Learner (EIL), by which agents can enable affective utility evaluation and adaptive learning procedure in multi-agent systems. We test EIL's performance in four games, including the iterated prisoners' dilemma, the ultimatum game and its multi-player variant, and the survival game. The results showed that EIL could significantly improve cooperation, promote altruism behaviors, and stimulate the sense of fairness in the equal mode, whereas increasing the trend of self-interest gradually in the process of switching to other modes. To sum up, our model illustrates that empathy can act as a virtual drive underlying cooperation and competition. This provides novel methods and insights in regulating behaviors in multi-agent systems, as well as artificial subjects in psychology and behavioral economics experiments.

show abstract

Section: Artificial Empathymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Artificial Empathy: A New Perspective for Analyzing and Designing Multi-Agent Systems

Chen

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Banerjee and Peng [21] referred policy dynamics based win or learn fast (PDWoLF) for some bimatrix game and general sum stochastic game. Zhang et al [22] studied the social-aware IGA (IGA-SA) by introducing social awareness into the strategy update process, if agents adopt rational learning strategies in the context of a repeated game and their strategies converge to the socially optimal outcomes of symmetric bimatrix games. In recent years, many scholars have designed different algorithms to analyze the power control game.…”

Section: Introductionmentioning

confidence: 99%

The WoLF-MFGA Algorithm for Equilibrium Analyses of Multiuser Power Control in Wireless Network

Liu

Jia

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The main goal of this paper is to explore strategic decision-making of multiuser power control in wireless network from the perspective of mean field game theory. First, we formulate the multiuser power control problem as a n -person noncooperative game and prove the existence of Nash equilibrium for multiuser power control game. Then, we design a new mean field gradient ascent with win or learn fast (WoLF-MFGA) algorithm by introducing mean field term into gradient ascent with the WoLF criterion algorithm. Subsequently, we investigate a sufficient condition of convergence for the proposed WoLF-MFGA algorithm. Furthermore, the WoLF-MFGA algorithm is used to achieve the Nash equilibrium of multiuser power control game, and some simulation results are given. Finally, for the sake of analyzing the convergence of the proposed algorithm, we investigate the sensitivity results of the WoLF-MFGA algorithm.

show abstract

“…The goal of MARL is determined by the type of task. The goal in a general-sum game is to converge to some type of equilibrium [7] or socially optimal outcomes [8], [9]. This type of learning is known as equilibrium-based MARL (EMARL).…”

Section: Introductionmentioning

confidence: 99%

A Cooperative Multi-Agent Reinforcement Learning Method Based on Coordination Degree

Cui

Zhang

2021

IEEE Access

View full text Add to dashboard Cite

Multi-agent reinforcement learning (MARL) has become a prevalent method for solving cooperative problems owing to its tractable implementation and task distribution. The goal of the MARL algorithms for fully cooperative scenarios is to obtain the optimal joint strategy that maximizes the expected common cumulative reward for all agents. However, to date, the analysis of MARL dynamics has focused on repeated games with few agents and actions. To this end, we propose a cooperative MARL algorithm based on the coordination degree (CMARL-CD) and analyze its dynamics in more general cases in which repeated games with more agents and actions are considered. Theoretical analysis shows that if the component action of every optimal joint action is unique, all optimal joint actions are asymptotically stable critical points. The CMARL-CD algorithm realizes coordination among agents without the need to estimate the global Q-value function. Each agent estimates the coordination degree of its own action, which represents the potential of being the optimal action. The efficacy of the CMARL-CD algorithm is studied through repeated games and stochastic games.

show abstract

SA-IGA: a multiagent reinforcement learning method towards socially optimal outcomes

Cited by 19 publications

References 28 publications

Artificial Empathy: A New Perspective for Analyzing and Designing Multi-Agent Systems

Artificial Empathy: A New Perspective for Analyzing and Designing Multi-Agent Systems

The WoLF-MFGA Algorithm for Equilibrium Analyses of Multiuser Power Control in Wireless Network

A Cooperative Multi-Agent Reinforcement Learning Method Based on Coordination Degree

Contact Info

Product

Resources

About