The dynamics of invariant object recognition in the human visual system

Işık, Leyla; Meyers, Ethan M.; Leibo, Joel Z.; Poggio, Tomaso

doi:10.1152/jn.00394.2013

Cited by 253 publications

(321 citation statements)

References 37 publications

(51 reference statements)

Supporting

Mentioning

289

Contrasting

Order By: Relevance

“…We found that the time course rose sharply after image onset, reaching significance at 50 ms (45-52 ms) and a peak at 97 ms (94-102 ms). This indicates that single scene images were discriminated early by visual representations, similar to single images with other visual content (Thorpe et al, 1996;Carlson et al, 2013;Cichy et al, 2014;Isik et al, 2014), suggesting a common source in early visual areas (Cichy et al, 2014).…”

Section: Neural Representations Of Single Scene Images Emerged Early mentioning

confidence: 86%

“…Using multivariate pattern classification (Carlson et al, 2013;Cichy et al, 2014;Isik et al, 2014) and representational similarity analysis (Kriegeskorte, 2008;Kriegeskorte and Kievit, 2013;Cichy et al, 2014) on millisecond-resolved magnetoencephalography data (MEG), we identified a marker of scene size around 250 ms, preceded by and distinct from an early signal for lower-level visual analysis of scene images at ~100ms. Furthermore, we demonstrated that the scene size marker was independent of both low-level image features (i.e.…”

Section: The Temporal Dynamics Of Spatial Layout Processingmentioning

confidence: 99%

See 1 more Smart Citation

Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks

Cichy

Khosla

Pantazis

et al. 2015

Preprint

View full text Add to dashboard Cite

Human scene recognition is a rapid multistep process evolving over time from single scene image to spatial layout processing. We used multivariate pattern analyses on magnetoencephalography (MEG) data to unravel the time course of this cortical process. Following an early signal for lowerlevel visual analysis of single scenes at ~100 ms, we found a marker of real-world scene size, i.e. spatial layout processing, at ~250 ms indexing neural representations robust to changes in unrelated scene properties and viewing conditions. For a quantitative model of how scene size representations may arise in the brain, we compared MEG data to a deep neural network model trained on scene classification. Representations of scene size emerged intrinsically in the model, and resolved emerging neural scene size representation. Together our data provide a first description of an electrophysiological signal for layout processing in humans, and suggest that deep neural networks are a promising framework to investigate how spatial layout representations emerge in the human brain.

show abstract

Section: Neural Representations Of Single Scene Images Emerged Early mentioning

confidence: 86%

Section: The Temporal Dynamics Of Spatial Layout Processingmentioning

confidence: 99%

Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks

Cichy

Khosla

Pantazis

et al. 2015

Preprint

View full text Add to dashboard Cite

show abstract

“…We recorded comprehensive brain activity with magnetoencephalography (MEG) in four adult human participants while they viewed face images from a large, carefully controlled set (91 face identities, with two facial expressions per identity; Fig. 1), with a sufficiently large number of trials for each face identity (104-112 trials per face identity, 9,464-10,192 trials per participant) to be able to evaluate the representation of individual face identities in each participant (26). We used MEG because it has excellent temporal resolution and sufficient spatial resolution for decoding of fine visual information from spatial patterns of neural activity (26,27).…”

Section: Significancementioning

confidence: 99%

“…1), with a sufficiently large number of trials for each face identity (104-112 trials per face identity, 9,464-10,192 trials per participant) to be able to evaluate the representation of individual face identities in each participant (26). We used MEG because it has excellent temporal resolution and sufficient spatial resolution for decoding of fine visual information from spatial patterns of neural activity (26,27). In each participant, we used an independent functional localizer task in MEG to identify face-selective regions in right lateral occipital cortex and right fusiform gyrus.…”

Section: Significancementioning

confidence: 99%

Spatiotemporal dynamics of similarity-based neural representations of facial identity

Vida

Nestor

Plaut

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Humans' remarkable ability to quickly and accurately discriminate among thousands of highly similar complex objects demands rapid and precise neural computations. To elucidate the process by which this is achieved, we used magnetoencephalography to measure spatiotemporal patterns of neural activity with high temporal resolution during visual discrimination among a large and carefully controlled set of faces. We also compared these neural data to lower level "image-based" and higher level "identitybased" model-based representations of our stimuli and to behavioral similarity judgments of our stimuli. Between ∼50 and 400 ms after stimulus onset, face-selective sources in right lateral occipital cortex and right fusiform gyrus and sources in a control region (left V1) yielded successful classification of facial identity. In all regions, early responses were more similar to the image-based representation than to the identity-based representation. In the face-selective regions only, responses were more similar to the identity-based representation at several time points after 200 ms. Behavioral responses were more similar to the identity-based representation than to the image-based representation, and their structure was predicted by responses in the face-selective regions. These results provide a temporally precise description of the transformation from low-to high-level representations of facial identity in human face-selective cortex and demonstrate that faceselective cortical regions represent multiple distinct types of information about face identity at different times over the first 500 ms after stimulus onset. These results have important implications for understanding the rapid emergence of fine-grained, highlevel representations of object identity, a computation essential to human visual expertise.face processing | magnetoencephalography | decoding | representational similarity analysis | face identity H umans can discriminate among thousands of highly similar and complex visual patterns, such as face identity, in less than half a second (1, 2). Efficient within-category discrimination of facial identity is important for real-world decisions (e.g., classifying a person as a friend or stranger) and social interactions. Progress has been made in elucidating the neural mechanisms underlying the discrimination of individual face identities in humans. Using fMRI, these studies demonstrate that individual face identities are represented by spatially distributed patterns of neural activity within occipitotemporal cortex (3-13). Because of the poor temporal resolution in fMRI studies (typically around 2 s), however, our understanding of the neural basis of discrimination among complex visual patterns in humans remains limited. For example, within a given region, different information relevant to discrimination may be represented at different times over the first few 100 ms after stimulus onset. However, current models of the neural basis of face recognition in humans do not typically allow for this possibility, becau...

show abstract

“…These feature differences are reflected in behavioral measures of perceptual similarity, such that within-and between-category perceptual similarity can be used to accurately predict the time it takes observers to categorize an object as animate or inanimate (Mohan and Arun, 2012). These perceptual differences likely contribute to MEG animacy decoding considering that object shape and perceptual similarity can be reliably decoded from MEG and EEG patterns (Isik et al, 2014;Coggan et al, 2016;Wardle et al, 2016). Furthermore, MEG animacy decoding strength is closely related, at the exemplar level, to categorization reaction time (Ritchie et al, 2015), likely reflecting the exemplar's perceptual typicality of the category it belongs to (Mohan and Arun, 2012).…”

Section: Introductionmentioning

confidence: 99%

MEG sensor patterns reflect perceptual but not categorical similarity of animate and inanimate objects

Proklova

Kaiser

Peelen³

2017

Preprint

View full text Add to dashboard Cite

High-level visual cortex shows a distinction between animate and inanimate objects, as revealed by fMRI. Recent studies have shown that object animacy can similarly be decoded from MEG sensor patterns. What object properties drive this decoding? Here, we disentangled the influence of perceptual and categorical properties by presenting perceptually matched objects that were easily recognizable as being animate or inanimate (e.g., snake and rope). In a series of behavioral experiments, three aspects of perceptual dissimilarity of these objects were quantified: overall dissimilarity, outline dissimilarity, and texture dissimilarity. Neural dissimilarity of MEG sensor patterns, collected in male and female human participants, was modeled using regression analysis, in which perceptual dissimilarity (taken from the behavioral experiments) and categorical dissimilarity served as predictors of neural dissimilarity. We found that perceptual dissimilarity was strongly reflected in MEG sensor patterns from 80 ms after stimulus onset, with separable contributions of outline and texture dissimilarity. Surprisingly, MEG patterns did not distinguish between animate and inanimate objects after controlling for perceptual dissimilarity. Nearly identical results were found in a second MEG experiment that required object recognition. These results indicate that MEG sensor patterns do not capture object animacy independently of perceptual differences between animate and inanimate objects. This is in contrast to results observed in fMRI using the same stimuli, task, and analysis approach, with fMRI showing a highly reliable categorical distinction in ventral temporal cortex even when controlling for perceptual dissimilarity. The discrepancy between MEG and fMRI precludes the straightforward integration of these imaging modalities. Significance statementRecent studies have shown that multivariate analysis of MEG sensor patterns allows for a detailed characterization of the time course of visual object processing, demonstrating that the neural representational space starts to become organized by object category (e.g., separating animate and inanimate objects) from around 150 ms after stimulus onset. It is unclear, however, whether this organization truly reflects a categorical distinction or whether it reflects uncontrolled differences in perceptual similarity (e.g., most animals have four legs). Here we find that MEG sensor patterns no longer distinguish between animate and inanimate objects when controlling for perceptual differences between objects (e.g., when comparing snake and rope). These results indicate that MEG sensor patterns are primarily sensitive to visual object properties.

show abstract

The dynamics of invariant object recognition in the human visual system

Cited by 253 publications

References 37 publications

Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks

Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks

Spatiotemporal dynamics of similarity-based neural representations of facial identity

MEG sensor patterns reflect perceptual but not categorical similarity of animate and inanimate objects

Contact Info

Product

Resources

About