Problems and Mysteries of the Many Languages of Thought

Mandelbaum, Eric; Dunham, Yarrow; Feiman, Roman; Firestone, Chaz; Green, E. J.; Harris, Daniel; Kibbe, Melissa M.; Kurdi, Benedek; Mylopoulos, Myrto; Shepherd, Joshua; Wellwood, Alexis; Porot, Nicolas; Quilty‐Dunn, Jake

doi:10.1111/cogs.13225

Cited by 12 publications

(11 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unsurprisingly, the architecture we favor can operate over representations in an LoT (Porot & Mandelbaum, 2020, 2022; Quilty-Dunn & Mandelbaum, 2018). We do not assume that logical operators of LoTs are just like those of formal or even natural languages; in fact, we agree with Canudas-Grabolosa et al , Cesana-Arlotti , and Wellwood & Hunter , who argue they differ (see also Mandelbaum et al, 2022; Porot, 2019).…”

Section: Lot and Developmentsupporting

confidence: 66%

The language-of-thought hypothesis as a working hypothesis in cognitive science

Quilty-Dunn,

Porot,

Mandelbaum

2023

Behav Brain Sci

View full text Add to dashboard Cite

The target article attempted to draw connections between broad swaths of evidence by noticing a common thread: Abstract, symbolic, compositional codes, that is, language-of-thoughts (LoTs). Commentators raised concerns about the evidence and offered fascinating extensions to areas we overlooked. Here we respond and highlight the many specific empirical questions to be answered in the next decade and beyond.

show abstract

Section: Lot and Developmentsupporting

confidence: 66%

The language-of-thought hypothesis as a working hypothesis in cognitive science

Quilty-Dunn,

Porot,

Mandelbaum

2023

Behav Brain Sci

View full text Add to dashboard Cite

show abstract

“…Nevertheless, LoT-like perceptual representations may not be fully language-like. While perception plausibly predicates properties of individuals (Quilty-Dunn & Green, 2021), it may lack the full expressive freedom of first-order logic (Camp, 2018), especially logical connectives needed for truth-functional completeness (Mandelbaum et al, 2022). Perception may be able to represent that an object is red but not that it is not red.…”

Section: Figure 1 Demonstrations Of Compositionality In Visual Percep...mentioning

confidence: 99%

Compositionality in visual perception

Hafri¹,

Green²,

Firestone³

2023

Preprint

View full text Add to dashboard Cite

Quilty-Dunn et al.’s wide-ranging defense of LoT argues that vision traffics in abstract, structured representational formats. We agree: Vision, like language, is compositional—just as words compose into phrases, many visual representations contain discrete constituents that combine in systematic ways. Here, we amass evidence extending this proposal, and explore its implications for how vision interfaces with the rest of the mind.

show abstract

“…However, modern neural networks, with no explicit inductive bias towards compositionality, have demonstrated successes on increasingly complex tasks. This raises the question: are these models succeeding by implementing compositional solutions under the hood (Mandelbaum et al, 2022)?…”

Section: Introductionmentioning

confidence: 99%

Break It Down: Evidence for Structural Compositionality in Neural Networks

Lepori¹,

Serre²,

Pavlick³

2023

Preprint

View full text Add to dashboard Cite

Many tasks can be described as compositions over subroutines. Though modern neural networks have achieved impressive performance on both vision and language tasks, we know little about the functions that they implement. One possibility is that neural networks implicitly break down complex tasks into subroutines, implement modular solutions to these subroutines, and compose them into an overall solution to a task -a property we term structural compositionality. Or they may simply learn to match new inputs to memorized representations, eliding task decomposition entirely. Here, we leverage model pruning techniques to investigate this question in both vision and language, across a variety of architectures, tasks, and pretraining regimens. Our results demonstrate that models oftentimes implement solutions to subroutines via modular subnetworks, which can be ablated while maintaining the functionality of other subroutines. This suggests that neural networks may be able to learn to exhibit compositionality, obviating the need for specialized symbolic mechanisms.

show abstract

Problems and Mysteries of the Many Languages of Thought

Cited by 12 publications

References 39 publications

The language-of-thought hypothesis as a working hypothesis in cognitive science

The language-of-thought hypothesis as a working hypothesis in cognitive science

Compositionality in visual perception

Break It Down: Evidence for Structural Compositionality in Neural Networks

Contact Info

Product

Resources

About