Cortical sensitivity to natural scene structure

Abstract

Natural scenes are inherently structured, with meaningful objects appearing in predictable locations. Human vision is tuned to this structure: When scene structure is purposefully jumbled, perception is strongly impaired. Here, we tested how such perceptual effects are reflected in neural sensitivity to scene structure. During separate fMRI and EEG experiments, participants passively viewed scenes whose spatial structure (i.e., the position of scene parts) and categorical structure (i.e., the content of scene parts) could be intact or jumbled. Using multivariate decoding, we show that spatial (but not categorical) scene structure profoundly impacts on cortical processing: Scene‐selective responses in occipital and parahippocampal cortices (fMRI) and after 255 ms (EEG) accurately differentiated between spatially intact and jumbled scenes. Importantly, this differentiation was more pronounced for upright than for inverted scenes, indicating genuine sensitivity to spatial structure rather than sensitivity to low‐level attributes. Our findings suggest that visual scene analysis is tightly linked to the spatial structure of our natural environments. This link between cortical processing and scene structure may be crucial for rapidly parsing naturalistic visual inputs.

Mapping the Scene and Object Processing Networks by Intracranial EEG

“…The P2 component, peaking at 220 ms, was described as an ERP marker for scene processing (Harel et al, 2016), showing the earliest discrimination between scenes and both objects and faces. In a follow-up parallel ERP and fMRI study, this component was localized to the scene-selective areas, OPA and PPA (Kaiser et al, 2020). But surprisingly, cluster S3 in our data, localized to the PPA, showed earlier discrimination at 164 ms. Kaiser et al (2020) also described earlier discrimination of spatially intact scenes starting at 55 ms and localized to V1, close to the time of discrimination of global scene properties at 84 ms in the Oz channel in scalp EEG study (Lowe et al, 2018).…”

“…In a follow-up parallel ERP and fMRI study, this component was localized to the scene-selective areas, OPA and PPA (Kaiser et al, 2020). But surprisingly, cluster S3 in our data, localized to the PPA, showed earlier discrimination at 164 ms. Kaiser et al (2020) also described earlier discrimination of spatially intact scenes starting at 55 ms and localized to V1, close to the time of discrimination of global scene properties at 84 ms in the Oz channel in scalp EEG study (Lowe et al, 2018). Results in this time range below 100 ms support conclusions from an earlier iEEG study showing decoding of five visual categories at around 100 ms after the stimulus (Liu et al, 2009).…”

Real-world structure facilitates the rapid emergence of scene category information in visual brain signals

“…Additionally, the stimulus set entailed scenes that were jumbled in their categorical content (with the individual scene parts stemming from different categories); these scenes were created to answer a different research question (see Kaiser et al, 2020a) and not used in the analyses reported in this paper. In both conditions relevant for 6 this paper, we used fragments from four different scenes to equate the presence of visual discontinuities between fragments.…”

“…Furthermore, 1,536 trials where the scene's categorical structure was altered have been recorded. This data has been analyzed elsewhere (see Kaiser et al, 2020a).…”

“…4 To address this question, we used a jumbling paradigm (Biederman, 1972;Biederman et al, 1974) that manipulates natural scenes' spatial structure: individual parts of the scene could either appear in their typical, "intact" positions or in atypical, "jumbled" positions ( Figure 1). In a recent neuroimaging study (Kaiser et al, 2020a), we employed this paradigm to show that in scene-selective visual cortex (fMRI) and after 250ms of vision (EEG), spatially intact scenes were represented differently from jumbled scenes. Here, we analyzed the EEG data from this jumbling paradigm to investigate whether the typical real-world structure, in contrast to an atypical jumbled structure, facilitates the visual representation of scene category.…”