The Spatiotemporal Neural Dynamics of Object Recognition for Natural Images and Line Drawings

Abstract

Drawings offer a simple and efficient way to communicate meaning. While line drawings capture only coarsely how objects look in reality, we still perceive them as resembling real-world objects. Previous work has shown that this perceived similarity is mirrored by shared neural representations for drawings and natural images, which suggests that similar mechanisms underlie the recognition of both. However, other work has proposed that representations of drawings and natural images become similar only after substantial processing has taken place, suggesting distinct mechanisms. To arbitrate between those alternatives, we measured brain responses resolved in space and time using fMRI and MEG, respectively, while human participants (female and male) viewed images of objects depicted as photographs, line drawings, or sketch-like drawings. Using multivariate decoding, we demonstrate that object category information emerged similarly fast and across overlapping regions in occipital, ventral-temporal and posterior parietal cortex for all types of depiction, yet with smaller effects at higher levels of visual abstraction. In addition, cross-decoding between depiction types revealed strong generalization of object category information from early processing stages on. Finally, by combining fMRI and MEG data using representational similarity analysis, we found that visual information traversed similar processing stages for all types of depiction, yet with an overall stronger representation for photographs. Together our results demonstrate broad commonalities in the neural dynamics of object recognition across types of depiction, thus providing clear evidence for shared neural mechanisms underlying recognition of natural object images and abstract drawings.SIGNIFICANCE STATEMENT:When we see a line drawing, we effortlessly recognize it as an object in the world despite its simple and abstract style. Here we asked to what extent this correspondence in perception is reflected in the brain. To answer this question, we measured how neural processing of objects depicted as photographs and line drawings with varying levels of detail (from natural images to abstract line drawings) evolves over space and time. We find broad commonalities in the spatiotemporal dynamics and the neural representations underlying the perception of photographs and even abstract drawings. These results indicate a shared basic mechanism supporting recognition of drawings and natural images.

Imaginary scenes are represented in cortical alpha activity

“…2b for a schematic). We chose to assess these shared alpha representations across the entire imagery period while maintaining temporal resolution for the perceptual data in order to increase the power of the analysis, since imagery representations were found to be relatively invariable across time (Corriveau et al, 2023; Dijkstra et al, 2018; Xie et al, 2020), whereas perceptual representations are thought to be more temporally variable (Carlson et al, 2011; Dijkstra et al, 2018; Singer et al, 2023). This yielded a time-resolved measure of shared representations in the alpha band between scene imagery and each stage in the processing hierarchy of scene perception, in which temporal representational variations index the procession of perceptual processing (King & Dehaene, 2014).…”

“…Shatek et al, 2019;Xie et al, 2020) and the multitaper approach typically increases the signal-to-noise ratio in frequency-resolved data at the expense of increased temporal and frequency smoothing (Cohen, 2014). In addition, we decided to omit temporal resolution of the frequency decomposition of the imagery data while maintaining it for the perception data in order to further boost power, since imagery representations have been shown to be relatively invariable across time (Corriveau et al, 2023;Dijkstra et al, 2018;Xie et al, 2020) while perceptual representations have been shown to be temporally variable as a function of the different processing stages in the visual hierarchy (Carlson et al, 2011;Dijkstra et al, 2018;King & Dehaene, 2014;Singer et al, 2023). Extracted frequencies ranged from 4-30 Hz, thus covering the theta (4-7 Hz), alpha (8-13 Hz) and beta (14-30 Hz) frequency bands.…”

Revisiting Snodgrass and Vanderwart in photograph form: The Keele Photo Stimulus Set (KPSS)

“…One theoretical question in the object recognition literature that this experiment cannot answer is: what do photographs truly add to recognition relative to line drawings (e.g., Biederman, 1987 ; Singer et al, 2023 ; Uttl et al, 2006 )? This is not a question that this study was designed to address as we did not directly compare Snodgrass and Vanderwart’s ( 1980 ) line drawings or Rossion and Pourtois’s ( 2004 ) textured drawings against our new greyscale and color photographic stimuli.…”

Spatiotemporal properties of common semantic categories for words and pictures

“…Therefore the final sample consisted of 31 participants (mean age pm SD = 21.77 pm 3.31; 21 female). The number of participants was selected to match the average number of participants used in previous studies that employed MVPA analysis (Leonardelli et al, 2019; Dirani and Pylkkänen, 2023; Singer et al, 2023). The study was conducted at the Center of Human Brain Health in Birmingham, United Kingdom.…”