Visual Hide and Seek

Autonomous agents must learn to collaborate. It is not scalable to develop a new centralized agent every time a task's difficulty outpaces a single agent's abilities. While multi-agent collaboration research has flourished in gridworld-like environments, relatively little work has considered visually rich domains. Addressing this, we introduce the novel task FurnMove in which agents work together to move a piece of furniture through a living room to a goal. Unlike existing tasks, FurnMove requires agents to coordinate at every timestep. We identify two challenges when training agents to complete FurnMove: existing decentralized action sampling procedures do not permit expressive joint action policies and, in tasks requiring close coordination, the number of failed actions dominates successful actions. To confront these challenges we introduce SYNC-policies (synchronize your actions coherently) and CORDIAL (coordination loss). Using SYNC-policies and CORDIAL, our agents achieve a 58% completion rate on FurnMove, an impressive absolute gain of 25 percentage points over competitive decentralized baselines. Our dataset, code, and pretrained models are available at https://unnat.github.io/cordial-sync.

show abstract

“…Similarly, a recent preprint [17] proposes a visual hide-and-seek task, where agents can move independently. Das et al .…”

Section: Related Workmentioning

confidence: 99%

A Cordial Sync: Going Beyond Marginal Policies for Multi-agent Embodied Tasks

Jain

Weihs

Kolve

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…In the early research, researchers have mainly studied the simple disembodied task for the multiagent setting [17], [18]. Recently, multi-agent decision models have been proposed to solve several multi-agent embodied tasks such as hide-and-seek task [19], hiding game [20] and the Quake III game [21]. Communication mechanism has also been studied [22] and applicated in a navigation task in which all agents need to find one same target through communication.…”

Section: Introductionmentioning

confidence: 99%

Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge

Xin-zhu¹,

Guo²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

In visual semantic navigation, the robot navigates to a target object with egocentric visual observations and the class label of the target is given. It is a meaningful task inspiring a surge of relevant research. However, most of the existing models are only effective for single-agent navigation, and a single agent has low efficiency and poor fault tolerance when completing more complicated tasks. Multi-agent collaboration can improve the efficiency and has strong application potentials. In this paper, we propose the multi-agent visual semantic navigation, in which multiple agents collaborate with others to find multiple target objects. It is a challenging task that requires agents to learn reasonable collaboration strategies to perform efficient exploration under the restrictions of communication bandwidth. We develop a hierarchical decision framework based on semantic mapping, scene prior knowledge, and communication mechanism to solve this task. The results of testing experiments in unseen scenes with both known objects and unknown objects illustrate the higher accuracy and efficiency of the proposed model compared with the single-agent model.

show abstract

Visual Hide and Seek

Cited by 2 publications

References 41 publications

A Cordial Sync: Going Beyond Marginal Policies for Multi-agent Embodied Tasks

A Cordial Sync: Going Beyond Marginal Policies for Multi-agent Embodied Tasks

Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge

Contact Info

Product

Resources

About