What is “Where”: Physical Reasoning Informs Object Location

Abstract

A central puzzle the visual system tries to solve is: “what is where?” While a great deal of research attempts to model object recognition (“what”), a comparatively smaller body of work seeks to model object location (“where”), especially in perceiving everyday objects. How do people locate an object, right now, in front of them? In three experiments collecting over 35,000 judgements on stimuli spanning different levels of realism (line drawings, real images, and crude forms), participants clicked “where” an object is, as if pointing to it. We modeled their responses with eight different methods, including both human response-based models (judgements of physical reasoning, spatial memory, free-response “click anywhere” judgements, and judgements of where people would grab the object), and image-based models (uniform distributions over the image, convex hull, saliency map, and medial axis). Physical reasoning was the best predictor of “where,” performing significantly better than even spatial memory and free-response judgements. Our results offer insight into the perception of object locations while also raising interesting questions about the relationship between physical reasoning and visual perception.

We are wanderers: Abstract geometry reflects spatial navigation.

“…Participants could have imagined a path or object with any geometry, especially considering that there was no actual navigation or object recognition involved in the task. Despite this freedom but consistent with other tasks’ use of an open-ended tapping procedure to elicit consistent spatial representations (Boger & Ullman, 2023; Firestone & Scholl, 2014), participants more often imagined open paths with distance and direction preserved and objects with global convex shape preserved. Human language may thus both naturally invoke the particular geometric representations inherent to navigation and object recognition given a basic description of the spatial context and also allow us easily to mentally wander between our different domains of foundational spatial knowledge about places and objects.…”

“…spatial representations(Boger & Ullman, 2023;Firestone & Scholl, 2014), participants more often imagined open paths with distance and direction preserved and objects with global convex shape preserved. Human language may thus both naturally invoke the particular geometric representations inherent to navigation and object recognition given a basic description of the spatial context and also allow us easily to mentally wander between our different domains of foundational spatial knowledge about places and objects.Sensitivity to this foundational geometric information is shared across cultures and through human development(Dehaene et al, …”