Inverse reinforcement learning for decentralized non-cooperative multiagent systems

Reddy, Tummalapalli Sudhamsh; Gopikrishna, Vamsikrishna; Záruba, Gergely; Huber, Manfred

doi:10.1109/icsmc.2012.6378020

Cited by 22 publications

(24 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the fact that most of the classical IRL techniques focus on learning reward functions for single-agent problems, recent proposals have begun to adapt IRL to learn multiagent reward functions. Supposing that the agents follow a Nash Equilibrium, Reddy et al (2012) propose a method for approximating the reward functions for all agents by computing them in a distributed manner. Natarajan et al (2010) apply IRL with a different purpose.…”

Section: Inverse Reinforcement Learningmentioning

confidence: 99%

“…Yet, IRL is a promising line of research, as training autonomous agents without explicit reward functions might be required for the development of general-purpose robots that receive instructions from laypeople. Despite these two issues, the recent trend on multiagent IRL might inspire new methods for influencing a group of agents to assume a collaborative behavior (Natarajan et al, 2010;Reddy et al, 2012;Lin et al, 2018).…”

Section: Inverse Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Silva¹,

Costa²

2019

jair

200

106

View full text Add to dashboard Cite

Multiagent Reinforcement Learning (RL) solves complex tasks that require coordination with other agents through autonomous exploration of the environment. However, learning a complex task from scratch is impractical due to the huge sample complexity of RL algorithms. For this reason, reusing knowledge that can come from previous experience or other agents is indispensable to scale up multiagent RL algorithms. This survey provides a unifying view of the literature on knowledge reuse in multiagent RL. We define a taxonomy of solutions for the general knowledge reuse problem, providing a comprehensive discussion of recent progress on knowledge reuse in Multiagent Systems (MAS) and of techniques for knowledge reuse across agents (that may be actuating in a shared environment or not). We aim at encouraging the community to work towards reusing all the knowledge sources available in a MAS. For that, we provide an in-depth discussion of current lines of research and open questions.

show abstract

Section: Inverse Reinforcement Learningmentioning

confidence: 99%

Section: Inverse Reinforcement Learningmentioning

confidence: 99%

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Silva¹,

Costa²

2019

jair

200

106

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

“…Multiple algorithms have been proposed for inverse reinforcement learning in multi-agent settings [1], [2], [11], [14], [15], [16], [17], [18], [19]. Both [14] and [15] extend the single-agent IRL algorithm of [20] to the multi-agent setting. [14] assumes that the problem can be described in terms of a centralized controller and a weighted cooperative reward function.…”

Section: Related Workmentioning

confidence: 99%

General-Sum Multi-Agent Continuous Inverse Optimal Control

Neumeyer

Oliehoek

Gavrila

2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Modelling possible future outcomes of robot-human interactions is of importance in the intelligent vehicle and mobile robotics domains. Knowing the reward function that explains the observed behaviour of a human agent is advantageous for modelling the behaviour with Markov Decision Processes (MDPs). However, learning the rewards that determine the observed actions from data is complicated by interactions. We present a novel inverse reinforcement learning (IRL) algorithm that can infer the reward function in multi-agent interactive scenarios. In particular, the agents may act boundedly rational (i.e., suboptimal), a characteristic that is typical for human decision making. Additionally, every agent optimizes its own reward function which makes it possible to address non-cooperative setups. In contrast to other methods, the algorithm does not rely on reinforcement learning during inference of the parameters of the reward function. We demonstrate that our proposed method accurately infers the ground truth reward function in two-agent interactive experiments 1 .

show abstract

“…Before introducing our algorithms, we review several existing approaches to MIRL and related problems. The first of these is a decentralized MIRL (d-MIRL) algorithm developed by Reddy et al [29]. This algorithm is decentralized in the sense that it infers agents' rewards one by one, rather than all at once.…”

Section: Conventional Mirl Approachesmentioning

confidence: 99%

Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games

Lin

Adams

Beling

2019

jair

View full text Add to dashboard Cite

This paper addresses the problem of multi-agent inverse reinforcement learning (MIRL) in a two-player general-sum stochastic game framework. Five variants of MIRL are considered: uCS-MIRL, advE-MIRL, cooE-MIRL, uCE-MIRL, and uNE-MIRL, each distinguished by its solution concept. Problem uCS-MIRL is a cooperative game in which the agents employ cooperative strategies that aim to maximize the total game value. In problem uCE-MIRL, agents are assumed to follow strategies that constitute a correlated equilibrium while maximizing total game value. Problem uNE-MIRL is similar to uCE-MIRL in total game value maximization, but it is assumed that the agents are playing a Nash equilibrium. Problems advE-MIRL and cooE-MIRL assume agents are playing an adversarial equilibrium and a coordination equilibrium, respectively. We propose novel approaches to address these five problems under the assumption that the game observer either knows or is able to accurate estimate the policies and solution concepts for players. For uCS-MIRL, we first develop a characteristic set of solutions ensuring that the observed bi-policy is a uCS and then apply a Bayesian inverse learning method. For uCE-MIRL, we develop a linear programming problem subject to constraints that define necessary and sufficient conditions for the observed policies to be correlated equilibria. The objective is to choose a solution that not only minimizes the total game value difference between the observed bi-policy and a local uCS, but also maximizes the scale of the solution. We apply a similar treatment to the problem of uNE-MIRL. The remaining two problems can be solved efficiently by taking advantage of solution uniqueness and setting up a convex optimization problem. Results are validated on various benchmark grid-world games.

show abstract

Inverse reinforcement learning for decentralized non-cooperative multiagent systems

Cited by 22 publications

References 7 publications

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

General-Sum Multi-Agent Continuous Inverse Optimal Control

Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games

Contact Info

Product

Resources

About