Visual analogy: Deep learning versus compositional models

Ichien, Nicholas; Liu, Qing; Fu, Shuhao; Holyoak, Keith J.; Yuille, Alan; Lu, Hongjing

doi:10.48550/arxiv.2105.07065

Cited by 2 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4 The failure of large language models in acquiring functional linguistic competence 4.1 LLMs are great at pretending to think Large text corpora contain a wealth of non-linguistic information, from mathematical and scientific facts (e.g., "two plus seven is nine") to factual knowledge (e.g., "the capital of Texas is Austin") to harmful stereotypes (e.g., "women belong in the kitchen"). This is not particularly surprising since even simple patterns of co-occurrence between words capture rich conceptual knowledge, including object properties [e.g., Grand et al, 2022, Huebner and Willits, 2018, Unger and Fisher, 2021, Utsumi, 2020, van Paridon et al, 2021, abstract analogies [Ichien et al, 2021], social biases [e.g., Bolukbasi et al, 2016, Caliskan et al, 2017, Lewis and Lupyan, 2020, and expert knowledge in specialized domains [e.g., Tshitoyan et al, 2019]. Moreover, statistical regularities extracted from language and from visual scenes exhibit a substantial degree of correspondence [Roads andLove, 2020, Sorscher et al, 2021], indicating that linguistic information can capture at least some aspects of experiential input [e.g., Abdou et al, 2021, Patel and.…”

Section: Interim Conclusionmentioning

confidence: 99%

Dissociating language and thought in large language models: a cognitive perspective

Mahowald¹,

Ivanova²,

Blank³

et al. 2023

Preprint

View full text Add to dashboard Cite

Short abstract (100 words): Large language models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their capabilities remain split. Here, we evaluate LLMs using a distinction between formal competence-knowledge of linguistic rules and patterns-and functional competence-understanding and using language in the world. We ground this distinction in human neuroscience, showing that these skills recruit different cognitive mechanisms. Although LLMs are close to mastering formal competence, they still fail at functional competence tasks, which often require drawing on non-linguistic capacities. In short, LLMs are good models of language but incomplete models of human thought.

show abstract

Section: Interim Conclusionmentioning

confidence: 99%

Dissociating language and thought in large language models: a cognitive perspective

Mahowald¹,

Ivanova²,

Blank³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…A new set of VAPs was also constructed in [168]. In contrast to V-PROM, matrices from [168] focus on renderings of realistic cars from the ShapeNet dataset [167], as shown in Fig. 18b.…”

Section: Avr-like Tasks For Representation Learningmentioning

confidence: 99%

“…The images vary in texture, shading and viewpoint. Using this dataset, the authors presented a case where a general segmentation model performed better than task-specific architectures trained solely on the automotive VAPs [168].…”

Section: Avr-like Tasks For Representation Learningmentioning

confidence: 99%

“…(b) Images of 3D car models (or their fragments) from the ShapeNet dataset[167] are arranged in an RPM-like grid. The trained model has to select an answer that best fits the analogy[168]. (c) Multiple subsequences are sampled from a video from UCF101[169] and the trained model has to detect the odd element -a subsequence with perturbed temporal order of video frames[170].…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Review of Emerging Research Directions in Abstract Visual Reasoning

Małkiński¹,

Mańdziuk²

2022

Preprint

View full text Add to dashboard Cite

Visual Reasoning (AVR) problems are commonly used to approximate human intelligence. They test the ability of applying previously gained knowledge, experience and skills in a completely new setting, which makes them particularly well-suited for this task. Recently, the AVR problems have become popular as a proxy to study machine intelligence, which has led to emergence of new distinct types of problems and multiple benchmark sets. In this work we review this emerging AVR research and propose a taxonomy to categorise the AVR tasks along 5 dimensions: input shapes, hidden rules, target task, cognitive function, and main challenge. The perspective taken in this survey allows to characterise AVR problems with respect to their shared and distinct properties, provides a unified view on the existing approaches for solving AVR tasks, shows how the AVR problems relate to practical applications, and outlines promising directions for future work. One of them refers to the observation that in the machine learning literature different tasks are considered in isolation, which is in the stark contrast with the way the AVR tasks are used to measure human intelligence, where multiple types of problems are combined within a single IQ test.

show abstract

Visual analogy: Deep learning versus compositional models

Cited by 2 publications

References 15 publications

Dissociating language and thought in large language models: a cognitive perspective

Dissociating language and thought in large language models: a cognitive perspective

A Review of Emerging Research Directions in Abstract Visual Reasoning

Contact Info

Product

Resources

About