A Puzzle concerning Compositionality in Machines

Nefdt, Ryan M.

doi:10.1007/s11023-020-09519-6

Cited by 12 publications

(8 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This in turn requires measures of how compositional an emergent language is (Andreas, 2019). Compositionality is a core notion in linguistics (Partee, 2004), but linguists' definitions assume full knowledge of primitive expressions and their combination rules, which we lack when analyzing emergent languages (Nefdt, 2020). Also, these definitions are categorical, whereas to compare emergent languages we need to quantify degrees of compositionality.…”

Section: Introductionmentioning

confidence: 99%

Compositionality and Generalization In Emergent Languages

Chaabouni¹,

Kharitonov²,

Bouchacourt³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Natural language allows us to refer to novel composite concepts by combining expressions denoting their parts according to systematic rules, a property known as compositionality. In this paper, we study whether the language emerging in deep multi-agent simulations possesses a similar ability to refer to novel primitive combinations, and whether it accomplishes this feat by strategies akin to human-language compositionality. Equipped with new ways to measure compositionality in emergent languages inspired by disentanglement in representation learning, we establish three main results. First, given sufficiently large input spaces, the emergent language will naturally develop the ability to refer to novel composite concepts. Second, there is no correlation between the degree of compositionality of an emergent language and its ability to generalize. Third, while compositionality is not necessary for generalization, it provides an advantage in terms of language transmission: The more compositional a language is, the more easily it will be picked up by new learners, even when the latter differ in architecture from the original agents. We conclude that compositionality does not arise from simple generalization pressure, but if an emergent language does chance upon it, it will be more likely to survive and thrive.

show abstract

Section: Introductionmentioning

confidence: 99%

Compositionality and Generalization In Emergent Languages

Chaabouni¹,

Kharitonov²,

Bouchacourt³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Computability does not entail compositionality. This point can be made through formal arguments (Pagin & Westerst ahl, 2010a) or concrete examples (see recent work using artificial neural networks; Baroni, 2020;Nefdt, 2020). Then, arguments that establish that meaning is or should be computable do not necessarily also establish the PSC as a logical consequence of computability.…”

Section: Why Compositionality?mentioning

confidence: 99%

Compositionality in a Parallel Architecture for Language Processing

Baggio

2021

Cognitive Science

View full text Add to dashboard Cite

Compositionality has been a central concept in linguistics and philosophy for decades, and it is increasingly prominent in many other areas of cognitive science. Its status, however, remains contentious. Here, I reassess the nature and scope of the principle of compositionality (Partee, 1995) from the perspective of psycholinguistics and cognitive neuroscience. First, I review classic arguments for compositionality and conclude that they fail to establish compositionality as a property of human language. Next, I state a new competence argument, acknowledging the fact that any competent user of a language L can assign to most expressions in L at least one meaning which is a function only of the meanings of the expression's parts and of its syntactic structure. I then discuss selected results from cognitive neuroscience, indicating that the human brain possesses the processing capacities presupposed by the competence argument. Finally, I outline a language processing architecture consistent with the neuroscience results, where semantic representations may be generated by a syntax-driven stream and by an "asyntactic" processing stream, jointly or independently. Compositionality is viewed as a constraint on computation in the former stream only.

show abstract

“…Nefdt [19] argues that the state of understanding of AI systems is such that "meaningful components" and compositional structures have not been established for deep neural networks. He does not extend his argument to the entire system, focusing instead on the mathematical "black box" that is a deep neural network.…”

Section: Heterogeneity In Representationmentioning

confidence: 99%

Scholarly AI system diagrams as an access point to mental models

Marshall¹,

Jay²,

Freitas³

2021

Preprint

View full text Add to dashboard Cite

Complex systems, such as Artificial Intelligence (AI) systems, are comprised of many interrelated components. In order to represent these systems, demonstrating the relations between components is essential. Perhaps because of this, diagrams, as "icons of relation", are a prevalent medium for signifying complex systems. Diagrams used to communicate AI system architectures are currently extremely varied. The diversity in diagrammatic conceptual modelling choices provides an opportunity to gain insight into the aspects which are being prioritised for communication. In this philosophical exploration of AI systems diagrams, we integrate theories of conceptual models, communication theory, and semiotics. We discuss consequences of standardised diagrammatic languages for AI systems, concluding that while we expect engineers implementing systems to benefit from standards, researchers would have a larger benefit from guidelines.

show abstract

A Puzzle concerning Compositionality in Machines

Cited by 12 publications

References 47 publications

Compositionality and Generalization In Emergent Languages

Compositionality and Generalization In Emergent Languages

Compositionality in a Parallel Architecture for Language Processing

Scholarly AI system diagrams as an access point to mental models

Contact Info

Product

Resources

About