The spatiotemporal neural dynamics of object location representations in the human brain

Abstract

To interact with objects in complex environments, we must know what they are and where they are in spite of challenging viewing conditions. Here, we investigated where, how and when representations of object location and category emerge in the human brain when objects appear on cluttered natural scene images using a combination of functional magnetic resonance imaging, electroencephalography and computational models. We found location representations to emerge along the ventral visual stream towards lateral occipital complex, mirrored by gradual emergence in deep neural networks. Time-resolved analysis suggested that computing object location representations involves recurrent processing in high-level visual cortex. Object category representations also emerged gradually along the ventral visual stream, with evidence for recurrent computations. These results resolve the spatiotemporal dynamics of the ventral visual stream that give rise to representations of where and what objects are present in a scene under challenging viewing conditions.

Independent spatiotemporal effects of spatial attention and background clutter on human object location representations

“…Equivalent testing in the dorsal visual stream revealed no significant main or interaction effect in any regions along the dorsal stream (Fig. 5F,G,H,I; Table 1), consistent with the observation that object location representations emerge rather along the ventral than the dorsal stream (Hong et al, 2016; Graumann et al, 2022).…”

“…The no clutter condition was a uniform gray background. In the high clutter condition, we selected 60 natural scene images from the Places365 database (http://places2.csail.mit.edu/download.html) that did not contain objects of the categories included in our experimental design (i.e., no animals, cars, faces, chairs) and were highly cluttered (as defined by 10 independent subject ratings; for methods and results see Graumann et al, 2022). We converted the images to grayscale and superimposed a circular aperture of 15 degrees.…”

“…For this we determined and compared the latencies of the classification peaks in the EEG time courses of both background conditions, assuming that the peaks represent the time points at which representations become most differentiable (DiCarlo and Cox, 2007). Our prediction was that location information would peak later in the high than in the no clutter condition, because dissecting objects from the background requires additional grouping and segmentation operations implemented in recurrent processing and thus increasing processing time (Groen et al, 2018;Seijdel et al, 2020Seijdel et al, , 2021Graumann et al, 2022).…”

“…2B) instead of four categories and locations. To characterize the effects of background and spatial attention on location representations in visual cortex, we defined regions-of-interest (ROIs) along the ventral stream, since HR predicted effects to emerge there based on previous studies (Hong et al, 2016;Graumann et al, 2022). We additionally included ROIs in the dorsal stream, since it has been implicated in visuospatial (Ungerleider and Haxby, 1994;Milner and Goodale, 2006;Kravitz et al, 2011;Groen et al, 2022) and attentional processing (Silver et al, 2005;Szczepanski et al, 2010;Sprague and Serences, 2013).…”

“…Our hypotheses are as follows. We set the stage by hypothesizing based on recent findings that the processing stage at which object location representations emerge depends on the object’s viewing conditions in particular its background (Graumann et al, 2022) independent of spatial attention. This replication hypothesis was termed HReplication (abbreviated HR; Fig.…”