Developing, evaluating and scaling learning agents in multi-agent environments

Gemp, Ian; Anthony, Thomas; Bachrach, Yoram; Bhoopchand, Avishkar; Bullard, Kalesha; Connor, Jerome T.; Dasagi, Vibhavari; Vylder, Bart De; Duéñez-Guzmán, Edgar A.; Élie, Romuald; Everett, Richard; Hennes, Daniel; Hughes, Edward; Khan, Mina; Lanctot, Marc; Larson, Kate; Lever, Guy; Liu, Siqi; Marris, Luke; McKee, Kevin R.; Müller, Paul; Pérolat, Julien; Strub, Florian; Tacchetti, Andrea; Tarassov, Eugene; Wang, Zhe; Tuyls, Karl

doi:10.3233/aic-220113

Cited by 3 publications

(3 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Coins [31,33,51,71] is a mixed-motive Markov game [54] played by n = 2 players (Fig. 2; see also Appendix A for full task details).…”

Section: Taskmentioning

confidence: 99%

Warmth and competence in human-agent cooperation

McKee,

Bai,

Fiske

2024

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

Interaction and cooperation with humans are overarching aspirations of artificial intelligence research. Recent studies demonstrate that AI agents trained with deep reinforcement learning are capable of collaborating with humans. These studies primarily evaluate human compatibility through “objective” metrics such as task performance, obscuring potential variation in the levels of trust and subjective preference that different agents garner. To better understand the factors shaping subjective preferences in human-agent cooperation, we train deep reinforcement learning agents in Coins, a two-player social dilemma. We recruit $$N = 501$$ N = 501 participants for a human-agent cooperation study and measure their impressions of the agents they encounter. Participants’ perceptions of warmth and competence predict their stated preferences for different agents, above and beyond objective performance metrics. Drawing inspiration from social science and biology research, we subsequently implement a new “partner choice” framework to elicit revealed preferences: after playing an episode with an agent, participants are asked whether they would like to play the next episode with the same agent or to play alone. As with stated preferences, social perception better predicts participants’ revealed preferences than does objective performance. Given these results, we recommend human-agent interaction researchers routinely incorporate the measurement of social perception and subjective preferences into their studies.

show abstract

“…Coins [31,33,51,71] is a mixed-motive Markov game [54] played by n = 2 players (Fig. 2; see also Appendix A for full task details).…”

Section: Taskmentioning

confidence: 99%

Warmth and competence in human-agent cooperation

McKee,

Bai,

Fiske

2024

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

show abstract

“…A few algorithms eschew training/testing but still cannot be considered fully decentralized since they require each player to be able to access the policies of other players (Foerster et al, 2018b; Jaques et al, 2019). Most algorithms in this class that can robustly find socially beneficial equilibria in collective action problems require public rewards (Eccles et al, 2019; Gemp et al, 2020; Hughes et al, 2018; McKee et al, 2020; Peysakhovich and Lerer, 2018; Wang et al, 2018) or the ability to redistribute rewards amongst agents’ (Lupu and Precup, 2020; Wang et al, 2021). This class of algorithms assumes that while they are learning all agents will have real-time access to one another’s rewards.…”

Section: Related Workmentioning

confidence: 99%

A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings

Vinitsky

Köster

Agapiou

et al. 2023

Collective Intelligence

View full text Add to dashboard Cite

Society is characterized by the presence of a variety of social norms: collective patterns of sanctioning that can prevent miscoordination and free-riding. Inspired by this, we aim to construct learning dynamics where potentially beneficial social norms can emerge. Since social norms are underpinned by sanctioning, we introduce a training regime where agents can access all sanctioning events but learning is otherwise decentralized. This setting is technologically interesting because sanctioning events may be the only available public signal in decentralized multi-agent systems where reward or policy-sharing is infeasible or undesirable. To achieve collective action in this setting, we construct an agent architecture containing a classifier module that categorizes observed behaviors as approved or disapproved, and a motivation to punish in accord with the group. We show that social norms emerge in multi-agent systems containing this agent and investigate the conditions under which this helps them achieve socially beneficial outcomes.

show abstract

“…• DeepMind [7] • Five AI [9] • Heriot-Watt University [13] • King's College London [2] • Teesside University [8] • University of Aberdeen [3] • University of Edinburgh [1] • University of Essex [11] • University of Lancaster [4]…”

mentioning

confidence: 99%

Multi-agent systems research in the United Kingdom

Albrecht

Wooldridge

2022

AIC

View full text Add to dashboard Cite

Multi-agent systems have been a core research topic in artificial intelligence for several decades. A multiagent system consists of multiple decision-making agents -which may be software-based AI systems, physicallyembodied robots, or humans -which must interact in a shared environment in pursuit of their goals. Multi-agent systems research spans a range of technical problems, such as how to design planning and learning algorithms which enable agents to achieve their goals; how to design multi-agent systems to incentivise certain behaviours in agents; how information is communicated and propagated among agents; and how norms, conventions, and roles may emerge in multi-agent systems. A vast array of applications have been addressed using multi-agent methodologies, including autonomous driving, multi-robot factories, automated trading, commercial games, automated tutoring, and robotic rescue teams.The purpose of this special issue is to showcase current multi-agent systems research led by university and industry groups based in the United Kingdom. Research groups and institutes in the UK which have significant activity in multi-agent systems research were invited to submit an article describing: (1) the technical problems in multi-agent systems tackled by the group (their core research agenda), including applications and industry collaboration; (2) the main approaches developed by the group and any key results achieved; and (3) important open challenges in multi-agent systems research from the perspective of the group.A large number of high-quality submissions were received, of which 14 were included for publication in the special issue. These articles represent a broad set of research topics within the field of multi-agent systems, showcasing the strength of contributions made by UK-based research groups in both universities and industry. We believe the open research problems discussed in each of the articles will provide a rich resource for researchers in this field, both new and old.Research groups from the following organisations are represented in the special issue (ordered alphabetically

show abstract

Developing, evaluating and scaling learning agents in multi-agent environments

Cited by 3 publications

References 46 publications

Warmth and competence in human-agent cooperation

Warmth and competence in human-agent cooperation

A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings

Multi-agent systems research in the United Kingdom

Contact Info

Product

Resources

About