Learning Coordination in Multi-Agent Systems Using Influence Value Reinforcement Learning

Abstract

In this paper authors propose a new paradigm for learning coordination in multi-agent systems. This approach is based on social interaction of people, specially in the fact that people communicate each other what they think about their actions and this opinion can influence the behavior of each other. It is proposed a model in which agents, into a multi-agent system, learns to coordinate actions giving opinions about actions of other agents and also being influenced with opinions of other agents about their actions.This paradigm was used to develop a modified version of the Q-learning algorithm. This algorithm was tested and compared with independent learning (IL) and joint action learning (JAL) in two single state problems with two agents. This approach shows to have more probability to converge to an optimal equilibrium than IL and JAL Q-learning algorithms. Also, it does not need to make an entire model of all joint actions like JAL algorithms.

Learning to Reach Optimal Equilibrium by Influence of Other Agents Opinion

“…As shown in figure 4, when exploration rate increases the Independent Learning Algorithm looses the capability of convergence to positions (1,3) and (3,3).…”

“…If they reach these final positions at the same time, they obtain a positive reward. When they reach the position (1, 3) at the same time they obtain 5 points and when they reach the position (3,3) (1,2), and finally the action go down leads the agent to position (3,2).…”

“…Our paradigm is based on social interaction of people [1]. When two persons interact, they communicate to each other what they think about their actions.…”

“…In previous work [1], authors shown that this paradigm outperforms traditional ones (independent learning and joint action learning) in general sum repetitive games. In this work, authors extend this model for application in two agent stochastic games and compare its performance with traditional paradigms in general sum games.…”

A solution for the Elevators Group Dispatch by Multiagent Reinforcement Learning

“…Uma questão crítica quando modelando o controle do despacho de grupos de elevadores para aprendizagem por reforçoé a explosão do espaço de estados. Um caminho trilhado para reduzir esta dificuldadeé modelar o grupo de elevadores como uma Sistema Multi-Agente (SMA) [Barrios- Aranibar and Gonçalves 2007]. Essa abordagem geralmente resulta em reduzir o armazenamento (espaço de estados) para cada agente, porém sacrificando em muito a curva velocidade de aprendizado [R. S. Sutton and A. G. Barto 2000].…”