Coding of navigational affordances in the human visual system

Abstract

A central component of spatial navigation is determining where one can and cannot go in the immediate environment. We used fMRI to test the hypothesis that the human visual system solves this problem by automatically identifying the navigational affordances of the local scene. Multivoxel pattern analyses showed that a scene-selective region of dorsal occipitoparietal cortex, known as the occipital place area, represents pathways for movement in scenes in a manner that is tolerant to variability in other visual features. These effects were found in two experiments: One using tightly controlled artificial environments as stimuli, the other using a diverse set of complex, natural scenes. A reconstruction analysis demonstrated that the population codes of the occipital place area could be used to predict the affordances of novel scenes. Taken together, these results reveal a previously unknown mechanism for perceiving the affordance structure of navigable space.scene-selective visual cortex | occipital place area | affordances | navigation | dorsal stream I t has long been hypothesized that perceptual systems are optimized for the processing of features that afford ecologically important behaviors (1, 2). This perspective has gained renewed support from recent work on action planning, which suggests that the action system continually prepares multiple, parallel plans that are appropriate for the environment (3, 4). If this view is correct, then sensory systems should be routinely engaged in identifying the potential of the environment for action, and they should explicitly and automatically encode these action affordances. Here we explore this idea for spatial navigation, a behavior that is essential for survival and ubiquitous among mobile organisms.A critical component of spatial navigation is the ability to understand where one can and cannot go in the local environment: for example, knowing that one can exit a room through a corridor or a doorway but not through a window or a painting on the wall. We reasoned that if perceptual systems routinely extract the parameters of the environment that delimit potential actions, then these navigational affordances should be automatically encoded during scene perception, even when subjects are not engaged in a navigational task.Previous work has shown that observers can determine the overall navigability of a scene-for example, whether it is possible to move through the scene or not-from a brief glance (5). However, no study has examined the coding of fine-grained navigational affordances, such as whether the direction one can move in the scene is to the left or to the right. Furthermore, only recently have investigators begun to characterize the affordance properties of scenes, and this work has focused not on navigational affordances but on more abstract behavioral events that can be used to define scene categories, such as cooking and sleeping (6). It therefore remains unknown whether navigational affordances play a fundamental role in the perception of visual scenes, an...

Rapid invariant encoding of scene layout in human OPA

“…kitchen versus woods) and scene identity (e.g. our kitchen versus someone else's) using a combination of geometry and texture cues, whereas OPA is involved in the analysis of the local scene elements (Bonner and Epstein, 2017;Julian et al, 2018;Kamps et al, 2016;Lowe et al, 2017). Scenes that we recognize as distinct places might have identical layout and it is the textures and object ensembles that provide the crucial cues to where we are.…”

“…Recent neuroimaging studies suggest a role for human OPA in detecting navigationally important cues from visual scenes. A functional magnetic resonance imaging (fMRI) study demonstrated that OPA encodes possible paths in a visual scene (Bonner and Epstein, 2017), and if processing in the OPA is temporarily disrupted using transcranial magnetic stimulation (TMS), a person's ability to use boundaries in a navigational task is impaired .…”

Large-scale dissociations between views of objects, scenes, and reachable-scale environments in visual cortex

“…The aim of this study was to characterize how the visual system responds to views of reachable environments relative to views of full-scale scenes and close-up singleton objects. We found that (1), reachable environments activate distinct response topographies from both scenes and objects, with reachspace preferences in posterior ventral cortex and throughout the dorsal visual stream, (2) peaks in this preference are present in consistent locations across participants, allowing us to define ROIs in the posterior collateral sulcus, dorsally in occipto-parietal cortex, and the superior parietal lobule; (3) the response topographies of reachspace preferences are maintained when low-level features are controlled; (4) reachspaces elicit dissociable activity in scene and object ROIs, driving these regions to an intermediate degree; (5) reachspace-preferring regions have peripheral biases and (6) have distinctly higher response to the presence of multiple isolated objects over near-scale spatial layout with minimal object content, a combination that is unique among the ROIs explored here.…”

“…trampoline, dresser, arranged in a 3x2 grid on a white background; (4) 6 objects with small real world size, e.g. mug, watch, arranged in a 3x2 grid on a white background, and presented at the same visual size as the previous image condition; (5) reachable environments with all objects removed except the support surface; (6) reachspaces containing only a single object on the support surface; (7) vertical reachspaces, where the disposition of objects was vertical rather than horizontal, e.g. shelves, peg-boards; (8) regular reachspaces, i.e.…”

The representational space of observed actions

“…the pathways for movements one may use to navigate through the scene) 25 . This profile is not only 193 important for visually-guided navigation and obstacle avoidance, but may also provide critical cues 194 indicating potential interactions with the local environment.…”