Compositionality and Generalization In Emergent Languages

Abstract

Natural language allows us to refer to novel composite concepts by combining expressions denoting their parts according to systematic rules, a property known as compositionality. In this paper, we study whether the language emerging in deep multi-agent simulations possesses a similar ability to refer to novel primitive combinations, and whether it accomplishes this feat by strategies akin to human-language compositionality. Equipped with new ways to measure compositionality in emergent languages inspired by disentanglement in representation learning, we establish three main results. First, given sufficiently large input spaces, the emergent language will naturally develop the ability to refer to novel composite concepts. Second, there is no correlation between the degree of compositionality of an emergent language and its ability to generalize. Third, while compositionality is not necessary for generalization, it provides an advantage in terms of language transmission: The more compositional a language is, the more easily it will be picked up by new learners, even when the latter differ in architecture from the original agents. We conclude that compositionality does not arise from simple generalization pressure, but if an emergent language does chance upon it, it will be more likely to survive and thrive.

Emergent Language Generalization and Acquisition Speed are not tied to Compositionality

“…Our concern is illustrated by the recent findings of Chaabouni et al (2020), who observed that the degree of compositionality of emergent languages is not correlated with the generalization capabilities of the agents that rely on them to solve a task. Indeed, lacking any specific pressure towards developing a (naïvely) compositional language, their agents were perfectly capable of developing generalizable but non-compositional communication systems.…”

“…Havrylov and Titov (2017) looked for symbol-position combina-tions that encode a single concept in an image, as a sign of a compositional behavior. A naïvely compositional language will maximize the two recently proposed compositionality measures of residual entropy (Resnick et al, 2019) and positional disentanglement (Chaabouni et al, 2020).…”

“…There is a recent spike of interest in studying the languages that emerge when artificial neural agents communicate to solve a common task (Foerster et al, 2016;Lazaridou et al, 2016;Havrylov and Titov, 2017). A good portion of such studies looks for traces of compositional structure in those languages, or even tries to inject such structure into them (Kottur et al, 2017;Choi et al, 2018;Mordatch and Abbeel, 2018;Andreas, 2019;Cogswell et al, 2019;Li and Bowling, 2019;Resnick et al, 2019;Chaabouni et al, 2020). Besides possibly providing insights on how compositionality emerged in natural language (Townsend et al, 2018), this emphasis is justified by the idea that a compositional language has various desirable properties.…”

Experience Grounds Language

“…Collaborative games have long served as a testbed for studying language (Werner and Dyer, 1991) and emergent communication (Schlangen, 2019a;Lazaridou et al, 2018;Chaabouni et al, 2020). Suhr et al (2019a) introduced an environment for evaluating language understanding in the service of a shared goal, and Andreas and Klein (2016) use a visual paradigm for studying pragmatics.…”

“LazImpa”: Lazy and Impatient neural agents learn to communicate efficiently

“…To facilitate such interaction, neural networks' emergent language should possess many natural-language-like properties. However, it has been shown that, even if these emergent languages lead to successful communication, they often do not bear core properties of natural language (Kottur et al, 2017;Bouchacourt and Baroni, 2018;Lazaridou et al, 2018;Chaabouni et al, 2020).…”