Teamwork with Limited Knowledge of Teammates

Barrett, Steven F.; Stone, Peter; Kraus, Sarit; Rosenfeld, Avi

doi:10.1609/aaai.v27i1.8659

Cited by 33 publications

(17 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, research efforts have been put on ad hoc interactions among agents, where agents need to interact without prior knowledge of the other agents (Stone et al 2010). They study both cooperative (Barrett et al 2013;Albrecht, Crandall, and Ramamoorthy 2016) and competitive (Southey et al 2012;Albrecht, Crandall, and Ramamoorthy 2016) scenarios, where agent type hypothesis and opponent modelling are used to handle the uncertainty. However, they do not consider the uncertainty in agent action observation and limited learning trials.…”

Section: Related Workmentioning

confidence: 99%

HogRider: Champion Agent of Microsoft Malmo Collaborative AI Challenge

Xiong

Chen

Zhao

et al. 2018

AAAI

View full text Add to dashboard Cite

It has been an open challenge for self-interested agents to make optimal sequential decisions in complex multiagent systems, where agents might achieve higher utility via collaboration. The Microsoft Malmo Collaborative AI Challenge (MCAC), which is designed to encourage research relating to various problems in Collaborative AI, takes the form of a Minecraft mini-game where players might work together to catch a pig or deviate from cooperation, for pursuing high scores to win the challenge. Various characteristics, such as complex interactions among agents, uncertainties, sequential decision making and limited learning trials all make it extremely challenging to find effective strategies. We present HogRider---the champion agent of MCAC in 2017 out of 81 teams from 26 countries. One key innovation of HogRider is a generalized agent type hypothesis framework to identify the behavior model of the other agents, which is demonstrated to be robust to observation uncertainty. On top of that, a second key innovation is a novel Q-learning approach to learn effective policies against each type of the collaborating agents. Various ideas are proposed to adapt traditional Q-learning to handle complexities in the challenge, including state-action abstraction to reduce problem scale, a warm start approach using human reasoning for addressing limited learning trials, and an active greedy strategy to balance exploitation-exploration. Challenge results show that HogRider outperforms all the other teams by a significant edge, in terms of both optimality and stability.

show abstract

Section: Related Workmentioning

confidence: 99%

HogRider: Champion Agent of Microsoft Malmo Collaborative AI Challenge

Xiong

Chen

Zhao

et al. 2018

AAAI

View full text Add to dashboard Cite

show abstract

“…While pre-coordinated multiagent teams are well studied, there has been less research into teams in which this precoordination is not available. This work builds on the ideas of Barrett et al (2013), specifically learning about past teammates and using this knowledge to quickly adapt to new teammates. However, that work focused on a simpler domain in the form of a grid world, while this work investigates a complex, simulated robotics domain.…”

Section: Related Workmentioning

confidence: 99%

“…This section describes PLASTIC-Policy, shown in Algorithm 1. PLASTIC-Policy is an extension of the algorithm proposed by Barrett et al (2013). Compared to that work, the main differences are 1) the use of a policy-based, modelfree method rather than a model-based approach and 2) the evaluations are on the much more complex domain of HFO.…”

Section: Plastic-policymentioning

confidence: 99%

“…However, Bayes' theorem may drop the posterior probability of a similar team to 0 for a single wrong prediction. Therefore, PLASTIC-Policy uses the approach of Barrett et al (2013) of updating these probabilities using the polynomial weights algorithm from regret minimization (Blum and Mansour 2007), shown on Lines 16-17, where η is empirically chosen to be 0.1.…”

Section: Policy Selectionmentioning

confidence: 99%

See 1 more Smart Citation

Cooperating with Unknown Teammates in Complex Domains: A Robot Soccer Case Study of Ad Hoc Teamwork

Barrett

Stone

2015

AAAI

Self Cite

View full text Add to dashboard Cite

Many scenarios require that robots work together as a team in order to effectively accomplish their tasks. However, pre-coordinating these teams may not always be possible given the growing number of companies and research labs creating these robots. Therefore, it is desirable for robots to be able to reason about ad hoc teamwork and adapt to new teammates on the fly. Past research on ad hoc teamwork has focused on relatively simple domains, but this paper demonstrates that agents can reason about ad hoc teamwork in complex scenarios. To handle these complex scenarios, we introduce a new algorithm, PLASTIC–Policy, that builds on an existing ad hoc teamwork approach. Specifically, PLASTIC– Policy learns policies to cooperate with past teammates and reuses these policies to quickly adapt to new teammates. This approach is tested in the 2D simulation soccer league of RoboCup using the half field offense task.

show abstract

“…However, in many domains we do not have such a model. The study of "ad-hoc" teamwork deals with multiagent teams with absence of information (Barrett et al 2013;Agmon and Stone 2012). They focus, however, on how a new agent must decide its behavior in order to cooperate with agents of unknown type, not on picking the best team.…”

Section: Related Workmentioning

confidence: 99%

Give a Hard Problem to a Diverse Team: Exploring Large Action Spaces

Marcolino

Jiang

et al. 2014

AAAI

View full text Add to dashboard Cite

Recent work has shown that diverse teams can outperform a uniform team made of copies of the best agent. However, there are fundamental questions that were not asked before. When should we use diverse or uniform teams? How does the performance change as the action space or the teams get larger? Hence, we present a new model of diversity for teams, that is more general than previous models. We prove that the performance of a diverse team improves as the size of the action space gets larger. Concerning the size of the diverse team, we show that the performance converges exponentially fast to the optimal one as we increase the number of agents. We present synthetic experiments that allow us to gain further insights: even though a diverse team outperforms a uniform team when the size of the action space increases, the uniform team will eventually again play better than the diverse team for a large enough action space. We verify our predictions in a system of Go playing agents, where we show a diverse team that improves in performance as the board size increases, and eventually overcomes a uniform team.

show abstract

Teamwork with Limited Knowledge of Teammates

Cited by 33 publications

References 24 publications

HogRider: Champion Agent of Microsoft Malmo Collaborative AI Challenge

HogRider: Champion Agent of Microsoft Malmo Collaborative AI Challenge

Cooperating with Unknown Teammates in Complex Domains: A Robot Soccer Case Study of Ad Hoc Teamwork

Give a Hard Problem to a Diverse Team: Exploring Large Action Spaces

Contact Info

Product

Resources

About