Emergent Communication in a Multi-Modal, Multi-Step Referential Game

Abstract

Inspired by previous work on emergent communication in referential games, we propose a novel multi-modal, multi-step referential game, where the sender and receiver have access to distinct modalities of an object, and their information exchange is bidirectional and of arbitrary duration. The multi-modal multi-step setting allows agents to develop an internal communication significantly closer to natural language, in that they share a single set of messages, and that the length of the conversation may vary according to the difficulty of the task. We examine these properties empirically using a dataset consisting of images and textual descriptions of mammals, where the agents are tasked with identifying the correct object. Our experiments indicate that a robust and efficient communication protocol emerges, where gradual information exchange informs better predictions and higher communication bandwidth improves generalization. INTRODUCTIONRecently, there has been a surge of work on neural network-based multi-agent systems that are capable of communicating with each other in order to solve a problem. Two distinct lines of research can be discerned. In the first one, communication is used as an essential tool for sharing information among multiple active agents in a reinforcement learning scenario (

PatchGame: Learning to Signal Mid-level Patches in Referential Games

“…To summarize, we propose PatchGame, a referential game formulation where given an image, the speaker sends discrete signal in terms of mid-level patches, and the listener embeds these symbols to match them with another view of the same image in the presence of distractors. Compared to previous works [22,30,45], we make the following key changes:…”

“…• Agents in the some of the prior works [22,30,45] have access to a pre-trained network, such as AlexNet [44] or VGG [69], for extracting features from images. In this work, the agents rely on training on a large scale image dataset, and invariance introduced by various image augmentations, to learn the language in a self-supervised way [53].…”

“…They used a fixed-sized message composed of a large vocabulary for their communication. Bouchacourt and Baroni [9], Evtimova et al [22], Havrylov and Titov [30] relax this assumption and allow communication via variable-length sequences. Havrylov and Titov [30] allows the speaker agent to use an LSTM [33] to construct a variable-length message.…”

“…A desirable characteristic of the speaker agent is that it should be able to encode "important" components of images with a variable length sequence of discrete symbols. Previous works [9,22,30] have achieved this by first converting the image into a continuous deterministic latent vector and then using an LSTM network [33] to generate a sequence of hidden states, and sample from this sequence of hidden state until a special end of sequence token (or maximum length) is reached. As observed by [30,46], in order to achieve the minimum loss, the model ends up always using the maximum allowable length.…”

Learning to Ground Multi-Agent Communication with Autoencoders

“…In contrast to non-referential games, constructing a communication protocol is critical to solving the task -where one can only arrive at a solution through communication. Referential games are referred to as a grounded learning environment, and therefore, communication in MARL has been studied mainly through the lens of referential games [13,26]. Lastly, ordinal tasks refer to a family of problems where the ordering of the actions is critical for solving the task.…”