Task context impacts visual object processing differentially across the cortex

Abstract

Perception reflects an integration of "bottom-up" (sensory-driven) and "top-down" (internally generated) signals. Although models of visual processing often emphasize the central role of feed-forward hierarchical processing, less is known about the impact of top-down signals on complex visual representations. Here, we investigated whether and how the observer's goals modulate object processing across the cortex. We examined responses elicited by a diverse set of objects under six distinct tasks, focusing on either physical (e.g., color) or conceptual properties (e.g., man-made). Critically, the same stimuli were presented in all tasks, allowing us to investigate how task impacts the neural representations of identical visual input. We found that task has an extensive and differential impact on object processing across the cortex. First, we found task-dependent representations in the ventral temporal and prefrontal cortex. In particular, although object identity could be decoded from the multivoxel response within task, there was a significant reduction in decoding across tasks. In contrast, the early visual cortex evidenced equivalent decoding within and across tasks, indicating task-independent representations. Second, task information was pervasive and present from the earliest stages of object processing. However, although the responses of the ventral temporal, prefrontal, and parietal cortex enabled decoding of both the type of task (physical/conceptual) and the specific task (e.g., color), the early visual cortex was not sensitive to type of task and could only be used to decode individual physical tasks. Thus, object processing is highly influenced by the behavioral goal of the observer, highlighting how top-down signals constrain and inform the formation of visual representations.object recognition | vision | fMRI | top-down processing | occipitotemporal cortex P erception reflects not only the external world but also our internal goals and biases. Even the simplest actions and decisions about visual objects require a complex integration between "top-down" (internally generated) and "bottom-up" (sensorydriven) signals (1). For example, the information used for object categorization depends on top-down signals arising from the spatial (2) or conceptual (3) context in which the object appears, the prior experience of the observers (4, 5), and the specific task (6, 7). Despite such strong behavioral evidence, the neural correlates of this integration remain unclear, both in terms of the cortical regions involved and the extent of the integration within those regions. Here, we investigate the impact of diverse behavioral goals on the neural architecture that supports object processing.Object recognition is known to depend on the ventral visual pathway, a set of interconnected cortical regions extending from early visual areas (e.g., V1/V2) into the anterior inferotemporal cortex (8). It has been argued that object processing along this pathway can largely be captured in feed-forward hierarchical frameworks wi...

Coding of Visual, Auditory, Rule, and Response Information in the Brain: 10 Years of Multivoxel Pattern Analysis

“…Accordingly, in human functional imaging, the strength of multivoxel codes in the MD system has been found to adjust according to task requirements, with perceptual discrimination increasing under conditions of high perceptual demand Woolgar, Hampshire, Thompson, & Duncan, 2011), rule discrimination increasing when rules are more complex (Woolgar, Afshar, Williams, & Rich, 2015), and a greater representation of visual objects that are at the focus of attention . These regions are also thought to make qualitatively different distinctions between visual stimuli in different task contexts (Harel, Kravitz, & Baker, 2014). The data presented here emphasize the extent of flexibility in these regions, suggesting that they are capable of representing task relevant information from visual, auditory, rule, and motor domains.…”

The Quest for the FFA and Where It Led

“…Although some studies reported higher FFA responses to objects of expertise than control objects (Gauthier et al, 2000;Xu, 2005), consistent with this hypothesis, these effects were small and many other studies failed to replicate them (Grill-Spector et al, 2004;Op de Beeck et al, 2006;Yue et al, 2006). Further, in all studies that have looked, expertise effects are not restricted to the FFA, but extend to multiple other brain regions (Gauthier et al, 2000;Harel et al, 2010;McGugin et al, 2012;Harel et al, 2014), as expected if these effects simply reflect greater attentional engagement by objects of expertise (Harel et al, 2010). Thus, there is no replicable evidence for a special linkage between the FFA (or face selectivity in general) and expertise.…”

The brain of the beholder: honouring individual representational idiosyncrasies

“…The representational geometry of a brain region has recently been shown to be somewhat influenced by top-down mechanisms such as attention and behavioural goals (Çukur, Nishimoto, Huth, & Gallant, 2013;Harel, Kravitz, & Baker, 2014;Rogers et al, 2006;Tyler et al, 2004). Çukur et al (2013) used fMRI to study the impact that searching for an object category during a natural movie has on the semantic representation measured in the brain.…”

“…Using voxel-wise modelling and regularised regression, the authors showed how attending one category distorted the semantic structure of the neural representations of both attended and unattended categories, with categoryattended expansion of the representational geometry at the Language, Cognition and Neuroscience 371 cost of a compression of the distant category (Çukur et al, 2013). In another study, Harel et al (2014) used fMRI to investigate how the neural representations in regions of the ventral temporal cortex vary as a function of task. They compared the brain-activity patterns for a single set of objects under six different tasks (fixation, colour, tilt, content, movement and size).…”