Feature-coding transitions to conjunction-coding with progression through human visual cortex

Abstract

Identifying an object and distinguishing it from similar items depends upon the ability to perceive its component parts as conjoined into a cohesive whole, but the brain mechanisms underlying this ability remain elusive. The ventral visual processing pathway in primates is organized hierarchically: Neuronal responses in its early stages are sensitive to the manipulation of simple visual features whereas neuronal responses in subsequent stages are tuned to increasingly complex stimulus attributes. It is widely assumed that feature-coding dominates in early visual cortex whereas later visual regions employ conjunction-coding in which object representations are different from the sum of their simple-feature parts. However, no study has demonstrated that putative object-level codes in higher visual cortex cannot be accounted for by feature-coding and that putative feature-codes in regions prior to ventral temporal cortex are not equally well characterized as object-level codes. Thus the existence of a transition from feature-to conjunctioncoding in visual cortex remains unconfirmed, and, if a transition does occur, its location remains unknown. By employing multivariate analysis of human functional imaging data, we measure both feature-coding and conjunction-coding directly, using the same set of visual stimuli, and pit them against each other to reveal the relative dominance of one versus the other throughout cortex. We provide the first demonstration of a transition from feature-coding in early visual cortex to conjunction-coding in both inferior temporal and posterior parietal cortices. This novel method enables the use of experimentally controlled stimulus features to investigate population-level feature-and conjunction-codes throughout human cortex.peer-reviewed) is the author/funder. All rights reserved. No reuse allowed without permission.The copyright holder for this preprint (which was not . http://dx.doi.org/10.1101/085381 doi: bioRxiv preprint first posted online Nov. 3, 2016; 3 Significance StatementModels of object recognition assume that feature-coded representations in early visual cortex are transformed by higher visual regions into an object-level code in which the whole is different from the sum of its parts, but direct evidence for this transformation is lacking. Cowell and Serences use a novel analysis of neuroimaging data to assess stimulus representations throughout human visual cortex, revealing a transition from feature-coding to conjunction-coding along both ventral and dorsal pathways. Representations in occipital cortex contain more information about spatial frequency and contour than about conjunctions of those features, whereas predominantly conjunction-specific information is observed in occipito-temporal, inferotemporal and superior parietal sites. The novel method will enable investigation of feature-and conjunction-coding in other sensory and motor modalities.peer-reviewed)

The locus of recognition memory signals in human cortex depends on the complexity of the memory representations

“…This is important, because LOC and PRC are the two regions among those we tested that are most clearly implicated in holistic visual object-processing as measured with perceptual tasks (e.g., Malach et al 1995; Grill-Spector et al 1998; Barense et al 2007). Third, the more general finding that, for Fribbles, conjunction memory signals were most strongly correlated with conjunction memory performance in anterior sites also accords with previous findings that complex conjunctive representations are found in anterior sites during perceptual tasks (e.g., Desimone et al 1984; Kobatake and Tanaka 1994; Malach et al 1995; Kanwisher et al 1997; Kriegeskorte et al 2008; Ostwald et al 2008; Drucker and Aguirre 2009; Cowell et al 2017). Fourth, for Scenes, evidence for a conjunction memory correlation was again negligible in visual cortex and steadily increased with anterior progression, reaching a maximum not in LOC or PRC, this time, but in HC.…”

“…This account proposes that the role of each brain region within the ventral visual-medial temporal lobe (MTL) pathway is best explained not by which cognitive function it performs, but by the type of information it represents. Building on well-established evidence for a hierarchy of increasingly complex stimulus representations in visual cortex (Hubel and Wiesel 1965; Felleman and Van Essen 1991; Kobatake and Tanaka 1994; Malach et al 1995; Kanwisher et al 1997; Kamitani and Tong 2005; Ostwald et al 2008; Henriksson et al 2008; Kriegeskorte et al 2008; Brouwer and Heeger 2009; Serences, Saproo, et al 2009; Drucker and Aguirre 2009; Cowell et al 2017), this account suggests that the representational hierarchy extends beyond visual cortex into the MTL. Thus, the pathway houses a graded continuum of representations, beginning with simple visual features such as oriented lines in V1 and culminating in complex multi-modal conjunctions corresponding to episodes or events in HC.…”

Content-based Dissociation of Hippocampal Involvement in Prediction

“…For instance, a triangle consists of the conjunction of three oriented lines intersecting at specific locations; if the lines were arranged differently, the triangle would cease to be. This definition of complexity is mirrored by the hierarchical organization of the visual processing pathway, with neural responses being tuned to simple visual features in early stages, and to increasingly complex (i.e., conjunctive) objects in later cortical stages (Cowell, Leger, & Serences, 2017). Visual processing in areas of the medial-temporal lobe (MTL) most directly connected to the hippocampus, such as perirhinal and parahippocampal cortices, is dominated by high-level objects and scenes, respectively (Martin, Douglas, Newsome, Man, & Barense, 2018;Murray, Bussey, & Saksida, 2007;Epstein & Kanwisher, 1998), as well as their spatial, temporal, and associative relations (Tsao et al, 2018;Garvert, Dolan, & Behrens, 2017;Hafting, Fyhn, Molden, Moser, & Moser, 2005).…”

An Emergent Population Code in Primary Auditory Cortex Supports Selective Attention to Spectral and Temporal Sound Features

“…However, no such "place" code exists for olfaction, since combinations of odorants comprise the "adequate stimulus" for olfaction and a two-dimensional sensory receptor surface ca not achieve a simple low-dimensional mapping of this combination space (Mathis et al, 2016;Herz et al, 2017). Like olfaction, auditory, somatic and visual perception also involve the combinations of many basic features (Walker et al, 2011;Gomez-Ramirez et al, 2014;Cowell et al, 2017;Franke et al, 2017;Lieber and Bensmaia, 2020). Therefore, when probed in high-dimensional conditions, all PSC fields may reflect a similar mechanism of representation to that observed in chemosensory areas.…”