Tangled Physics: Knots as a challenge for physical scene understanding

Croom, Sholei; Firestone, Chaz

doi:10.1167/jov.21.9.2653

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The extraction of force-dynamic relations in automatic visual processing may also have implications for how observers intuit physical states of the world (e.g., what will move where; Kubricht et al, 2017 ; McCloskey et al, 1980 ; Ullman et al, 2017 ). Although some research suggests that such physical predictions are made via mental simulations that utilize a kind of “physics engine” in the head (e.g., Battaglia et al, 2013 ), other work proposes theoretical and empirical limits on such processes (e.g., Croom & Firestone, 2021 ; Davis & Marcus, 2015 ; Ludwin-Peery et al, 2019 ), leaving open how the mind accomplishes seemingly effortless inference about physical situations. Our work suggests that visual processing may automatically classify configurations of objects into abstract relational types ( in or on )—perhaps even when the relations involve novel combinations of objects (Garnelo & Shanahan, 2019 ) or when the objects are totally unfamiliar or underspecified (Davis et al, 2017 ).…”

Section: Discussionmentioning

confidence: 99%

A Phone in a Basket Looks Like a Knife in a Cup: Role-Filler Independence in Visual Processing

Hafri,

Bonner,

Landau

et al. 2024

Open Mind

Self Cite

View full text Add to dashboard Cite

When a piece of fruit is in a bowl, and the bowl is on a table, we appreciate not only the individual objects and their features, but also the relations containment and support, which abstract away from the particular objects involved. Independent representation of roles (e.g., containers vs. supporters) and “fillers” of those roles (e.g., bowls vs. cups, tables vs. chairs) is a core principle of language and higher-level reasoning. But does such role-filler independence also arise in automatic visual processing? Here, we show that it does, by exploring a surprising error that such independence can produce. In four experiments, participants saw a stream of images containing different objects arranged in force-dynamic relations—e.g., a phone contained in a basket, a marker resting on a garbage can, or a knife sitting in a cup. Participants had to respond to a single target image (e.g., a phone in a basket) within a stream of distractors presented under time constraints. Surprisingly, even though participants completed this task quickly and accurately, they false-alarmed more often to images matching the target’s relational category than to those that did not—even when those images involved completely different objects. In other words, participants searching for a phone in a basket were more likely to mistakenly respond to a knife in a cup than to a marker on a garbage can. Follow-up experiments ruled out strategic responses and also controlled for various confounding image features. We suggest that visual processing represents relations abstractly, in ways that separate roles from fillers.

show abstract

Section: Discussionmentioning

confidence: 99%

A Phone in a Basket Looks Like a Knife in a Cup: Role-Filler Independence in Visual Processing

Hafri,

Bonner,

Landau

et al. 2024

Open Mind

Self Cite

View full text Add to dashboard Cite

show abstract

“…Anonymized raw and analyzed data are available on the Open Science Framework ( https://osf.io/wndkg/ ) ( 18 ).…”

Section: Data Materials and Software Availabilitymentioning

confidence: 99%

Seeing and understanding epistemic actions

Croom,

Zhou,

Firestone

2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Many actions have instrumental aims, in which we move our bodies to achieve a physical outcome in the environment. However, we also perform actions with epistemic aims, in which we move our bodies to acquire information and learn about the world. A large literature on action recognition investigates how observers represent and understand the former class of actions; but what about the latter class? Can one person tell, just by observing another person’s movements, what they are trying to learn? Here, five experiments exploreepistemic action understanding. We filmed volunteers playing a “physics game” consisting of two rounds: Players shook an opaque box and attempted to determine i) the number of objects hidden inside, or ii) the shape of the objects inside. Then, independent subjects watched these videos and were asked to determine which videos came from which round: Who was shaking for number and who was shaking for shape? Across several variations, observers successfully determined what an actor was trying to learn, based only on their actions (i.e., how they shook the box)—even when the box’s contents were identical across rounds. These results demonstrate that humans can infer epistemic intent from physical behaviors, adding a new dimension to research on action understanding.

show abstract

Tangled Physics: Knots as a challenge for physical scene understanding

Cited by 2 publications

References 0 publications

A Phone in a Basket Looks Like a Knife in a Cup: Role-Filler Independence in Visual Processing

A Phone in a Basket Looks Like a Knife in a Cup: Role-Filler Independence in Visual Processing

Seeing and understanding epistemic actions

Contact Info

Product

Resources

About