A Comprehensive Survey of Multiagent Reinforcement Learning

Buşoniu, Lucian; Babuška, Robert; Schutter, Bart De

doi:10.1109/tsmcc.2007.913919

Cited by 1,598 publications

(929 citation statements)

References 92 publications

Supporting

Mentioning

911

Contrasting

Unclassified

Order By: Relevance

“…For the computation of CEs, except for the linear programming approach as we employed, multiagent learning, see e.g., [Shoham et al, 2007;Busoniu et al, 2008] for comprehensive reviews, focuses on extending the reinforcement learning approaches to a multiagent setting. In particular, [Greenwald and Hall, 2003] presents an extension of the Q-learning algorithm to find a CE.…”

Section: Related Workmentioning

confidence: 99%

ATL Strategic Reasoning Meets Correlated Equilibrium

Huang

Ruan

2017

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

This paper is motivated by analysing a Google selfdriving car accident, i.e., the car hit a bus, with the framework and the tools of strategic reasoning by model checking. First of all, we find that existing ATL model checking may find a solution to the accident with irrational joint strategy of the bus and the car. This leads to a restriction of treating both the bus and the car as rational agents, by which their joint strategy is an equilibrium of certain solution concepts. Second, we find that a randomly-selected joint strategy from the set of equilibria may result in the collision of the two agents, i.e., the accident. Based on these, we suggest taking Correlated Equilibrium (CE) as agents' joint strategy and optimising over the utilitarian value which is the expected sum of the agents' total rewards. The language ATL is extended with two new modalities to express the existence of a CE and a unique CE, respectively. We implement the extension into a software model checker and use the tool to analyse the examples in the paper. We also study the complexity of the model checking problem.

show abstract

Section: Related Workmentioning

confidence: 99%

ATL Strategic Reasoning Meets Correlated Equilibrium

Huang

Ruan

2017

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…Besides single-agent reinforcement learning, MARL has strong connections with game theory, evolutionary computation, and optimization theory. We refer the reader to [4] for a survey of the works in this area and discuss some relevant ideas here. Many multi-agent algorithms exist for di↵erent tasks which range from fully cooperative setting [22,16,48] to fully competitive setting [23,15].…”

Section: Related Workmentioning

confidence: 99%

Anticipatory Planning for Human-Robot Teams

Koppula

Jain

Saxena

2015

Experimental Robotics

View full text Add to dashboard Cite

Abstract. When robots work alongside humans for performing collaborative tasks, they need to be able to anticipate human's future actions and plan appropriate actions. The tasks we consider are performed in contextually-rich environments containing objects, and there is a large variation in the way humans perform these tasks. We use a graphical model to represent the state-space, where we model the humans through their low-level kinematics as well as their high-level intent, and model their interactions with the objects through physically-grounded object a↵ordances. This allows our model to anticipate a belief about possible future human actions, and we model the human's and robot's behavior through an MDP in this rich state-space. We further discuss that due to perception errors and the limitations of the model, the human may not take the optimal action and therefore we present robot's anticipatory planning with di↵erent behaviors of the human within the model's scope. In experiments on Cornell Activity Dataset, we show that our method performs better than various baselines for collaborative planning.

show abstract

“…Several different MARL algorithms have been proposed to find optimal policies in Markov games including competitive, fully cooperative, and general ones (Busniu, Babuska, & Schutter, 2008). Most of these algorithms seek to find the equilibrium policy.…”

Section: Related Workmentioning

confidence: 99%

“…Thus, they are able to build up the game structure. A detailed study of different algorithms can be found in (Busniu et al, 2008).…”

Section: Related Workmentioning

confidence: 99%

Speeding up learning automata based multi agent systems using the concepts of stigmergy and entropy

Masoumi

Meybodi

2011

Expert Systems with Applications

View full text Add to dashboard Cite

a b s t r a c tLearning automata (LA) were recently shown to be valuable tools for designing Multi-Agent Reinforcement Learning algorithms and are able to control the stochastic games. In this paper, the concepts of stigmergy and entropy are imported into learning automata based multi-agent systems with the purpose of providing a simple framework for interaction and coordination in multi-agent systems and speeding up the learning process. The multi-agent system considered in this paper is designed to find optimal policies in Markov games. We consider several dummy agents that walk around in the states of the environment, make local learning automaton active, and bring information so that the involved learning automaton can update their local state. The entropy of the probability vector for the learning automata of the next state is used to determine reward or penalty for the actions of learning automata. The experimental results have shown that in terms of the speed of reaching the optimal policy, the proposed algorithm has better learning performance than other learning algorithms.

show abstract

A Comprehensive Survey of Multiagent Reinforcement Learning

Cited by 1,598 publications

References 92 publications

ATL Strategic Reasoning Meets Correlated Equilibrium

ATL Strategic Reasoning Meets Correlated Equilibrium

Anticipatory Planning for Human-Robot Teams

Speeding up learning automata based multi agent systems using the concepts of stigmergy and entropy

Contact Info

Product

Resources

About