Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a “face” area?

Hanson, Stephen José; Matsuka, Toshihiko; Haxby, James V.

doi:10.1016/j.neuroimage.2004.05.020

Cited by 245 publications

(222 citation statements)

References 22 publications

Supporting

Mentioning

208

Contrasting

Order By: Relevance

“…Finally, connectionists have long employed hidden layers in their neural networks to mediate non-linear correspondences between input and output. Hanson et al (2004) proposed a neural network classifier with hidden units to account for brain activation patterns, but the learned hidden units are difficult to interpret in terms of an intermediate semantic representation.…”

Section: Contents Lists Available At Sciencedirectmentioning

confidence: 99%

Quantitative modeling of the neural representation of objects: How semantic feature norms can account for fMRI activation

2011

View full text Add to dashboard Cite

Recent multivariate analyses of fMRI activation have shown that discriminative classifiers such as Support Vector Machines (SVM) are capable of decoding fMRI-sensed neural states associated with the visual presentation of categories of various objects. However, the lack of a generative model of neural activity limits the generality of these discriminative classifiers for understanding the underlying neural representation. In this study, we propose a generative classifier that models the hidden factors that underpin the neural representation of objects, using a multivariate multiple linear regression model. The results indicate that object features derived from an independent behavioral feature norming study can explain a significant portion of the systematic variance in the neural activity observed in an object-contemplation task. Furthermore, the resulting regression model is useful for classifying a previously unseen neural activation vector, indicating that the distributed pattern of neural activities encodes sufficient signal to discriminate differences among stimuli. More importantly, there appears to be a double dissociation between the two classifier approaches and within-versus between-participants generalization. Whereas an SVM-based discriminative classifier achieves the best classification accuracy in within-participants analysis, the generative classifier outperforms an SVM-based model which does not utilize such intermediate representations in between-participants analysis. This pattern of results suggests the SVM-based classifier may be picking up some idiosyncratic patterns that do not generalize well across participants and that good generalization across participants may require broad, large-scale patterns that are used in our set of intermediate semantic features. Finally, this intermediate representation allows us to extrapolate the model of the neural activity to previously unseen words, which cannot be done with a discriminative classifier.

show abstract

Section: Contents Lists Available At Sciencedirectmentioning

confidence: 99%

Quantitative modeling of the neural representation of objects: How semantic feature norms can account for fMRI activation

2011

View full text Add to dashboard Cite

show abstract

“…An ever increasing number of studies have succeeded in detecting patterns of brain activity that reflect the identity or the semantic category of pictures (Haxby et al, 2001;Hanson et al, 2004;Shinkareva et al, 2008;Connolly et al, 2012;Mur et al, 2012;Bruffaerts et al, 2013a), written words Just et al, 2010), written words and pictures (Shinkareva et al, 2011;Simanova et al, 2012;Bruffaerts et al, 2013b;Devereux et al, 2013;Fairhall and Caramazza, 2013), written and spoken words (Akama et al, 2012;Simanova et al, 2012) as well as natural sounds (Simanova et al, 2012).…”

Section: Introductionmentioning

confidence: 99%

Left perirhinal cortex codes for similarity in meaning between written words: Comparison with auditory word input

et al. 2015

View full text Add to dashboard Cite

This is a pre-copy-editing, author-produced PDF of an article accepted for publication in Neuropsychologia following peer review. The definitive publisher-authenticated version (Liuzzi A.G., Bruffaerts R., Dupont P., Adamczuk K., Peeters R., De Deyne S., Storms G., Vandenberghe R. AbstractLeft perirhinal cortex has been previously implicated in associative coding. According to a recent experiment, the similarity of perirhinal fMRI response patterns to written concrete words is higher for words which are more similar in their meaning. If left perirhinal cortex functions as an amodal semantic hub, one would predict that this semantic similarity effect would extend to the spoken modality. We conducted an event-related fMRI experiment and evaluated whether a same semantic similarity effect could be obtained for spoken as for written words. Twenty healthy subjects performed a property verification task in either the written or the spoken modality. Words corresponded to concrete animate entities for which extensive feature generation was available from more than 1000 subjects. From these feature generation data, a concept-feature matrix was derived which formed the basis of a cosine similarity matrix between the entities reflecting their similarity in meaning (called the "semantic cossimilarity matrix"). Independently, we calculated a cosine similarity matrix between the left perirhinal fMRI activity patterns evoked by the words (called the "fMRI cossimilarity matrix"). Next, the similarity was determined between the semantic cossimilarity matrix and the fMRI cossimilarity matrix. This was done for written and spoken words pooled, for written words only, for spoken words only, as well as for crossmodal pairs. Only for written words did the fMRI cossimilarity matrix correlate with the semantic cossimilarity matrix. Contrary to our prediction, we did not find any such effect for auditory word input nor did we find cross-modal effects in perirhinal cortex between written and auditory words. Our findings situate the contribution of left perirhinal cortex to word processing at the top of the visual processing pathway, rather than at an amodal stage where visual and auditory word processing pathways have already converged.

show abstract

“…This approach has the potential to be particularly useful in determining how semantic information about objects is represented in the cerebral cortex. Using multivoxel pattern analysis, previous studies succeeded in identifying the cognitive states associated with viewing categories of objects [Carlson et al, 2003;Cox and Savoy, 2003;Hanson and Halchenko, 2007;Hanson et al, 2004;Haxby et al, 2001;O'Toole et al, 2005;Polyn et al, 2005]. Moreover, the category of an object that a participant was viewing [Shinkareva et al, 2008[Shinkareva et al, , 2011 or a concrete noun that a participant was reading [Just et al, 2010;Shinkareva et al, 2011] can be identified based only on other participants' characteristic neural activation patterns, establishing the commonality in how different people's brains represent the same object.…”

Section: Introductionmentioning

confidence: 99%

Exploring commonalities across participants in the neural representation of objects

Shinkareva

Malave

Just

et al. 2011

Human Brain Mapping

View full text Add to dashboard Cite

The question of whether the neural encodings of objects are similar across different people is one of the key questions in cognitive neuroscience. This article examines the commonalities in the internal representation of objects, as measured with fMRI, across individuals in two complementary ways. First, we examine the commonalities in the internal representation of objects across people at the level of interobject distances, derived from whole brain fMRI data, and second, at the level of spatially localized anatomical brain regions that contain sufficient information for identification of object categories, without making the assumption that their voxel patterns are spatially matched in a common space. We examine the commonalities in internal representation of objects on 3T fMRI data collected while participants viewed line drawings depicting various tools and dwellings. This exploratory study revealed the extent to which the representation of individual concepts, and their mutual similarity, is shared across participants. Hum Brain Mapp 33:1375-1383,

show abstract

Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a “face” area?

Cited by 245 publications

References 22 publications

Quantitative modeling of the neural representation of objects: How semantic feature norms can account for fMRI activation

Quantitative modeling of the neural representation of objects: How semantic feature norms can account for fMRI activation

Left perirhinal cortex codes for similarity in meaning between written words: Comparison with auditory word input

Exploring commonalities across participants in the neural representation of objects

Contact Info

Product

Resources

About