Representational Similarity Analysis Reveals Commonalities and Differences in the Semantic Processing of Words and Objects

Devereux, Barry; Clarke, Alex; Marouchos, Andreas; Tyler, Lorraine K.

doi:10.1523/jneurosci.3809-13.2013

Cited by 173 publications

(178 citation statements)

References 57 publications

Supporting

Mentioning

156

Contrasting

Order By: Relevance

“…This method is therefore wellsuited for assessing neural overlap between semantic representations (23)(24)(25), as the degree of overlap should be directly reflected in the representational similarity. We used this approach to measure the degree of neural overlap between each set of DRM words and their related lure word (Fig.…”

Section: Resultsmentioning

confidence: 99%

Semantic representations in the temporal pole predict false memories

Chadwick

Anjum

Kumaran

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Recent advances in neuroscience have given us unprecedented insight into the neural mechanisms of false memory, showing that artificial memories can be inserted into the memory cells of the hippocampus in a way that is indistinguishable from true memories. However, this alone is not enough to explain how false memories can arise naturally in the course of our daily lives. Cognitive psychology has demonstrated that many instances of false memory, both in the laboratory and the real world, can be attributed to semantic interference. Whereas previous studies have found that a diverse set of regions show some involvement in semantic false memory, none have revealed the nature of the semantic representations underpinning the phenomenon. Here we use fMRI with representational similarity analysis to search for a neural code consistent with semantic false memory. We find clear evidence that false memories emerge from a similarity-based neural code in the temporal pole, a region that has been called the "semantic hub" of the brain. We further show that each individual has a partially unique semantic code within the temporal pole, and this unique code can predict idiosyncratic patterns of memory errors. Finally, we show that the same neural code can also predict variation in true-memory performance, consistent with an adaptive perspective on false memory. Taken together, our findings reveal the underlying structure of neural representations of semantic knowledge, and how this semantic structure can both enhance and distort our memories.false memory | semantic | temporal pole | fMRI | pattern similarity E ach of us has a vast store of semantic knowledge that we apply to incoming sensory data to extract meaning from the world around us. Semantic representations are capable of capturing important structural features of the world at many different levels of abstraction, which allows for rapid and flexible responses to a diverse array of environmental challenges. This preexisting knowledge structure guides ongoing cognition, which usually aids performance, but under some circumstances can lead us into error (1-3). A striking example is the widely studied DRM (Deese, Roediger, and McDermott) false-memory illusion (4, 5). In a typical DRM task, subjects are asked to memorize a set of words such as "snow," "winter," "ice," and "warm." After a delay, subjects will typically falsely remember having seen the semantically related word "cold." It is widely agreed that this memory illusion is driven by the semantic relatedness between words contained in the encoding list (e.g., "snow") and falsely remembered words that were not actually presented (e.g., "cold"). As such, it is thought that each list item automatically, but weakly, activates the semantically related concept (Fig. 1A). This activation leads to memory confusion, either through a cumulative priming of the related lure (5, 6) or the encoding of the semantic overlap as a "gist" memory (3), resulting in a false memory unless the error is detected by some internal monitoring p...

show abstract

Section: Resultsmentioning

confidence: 99%

Semantic representations in the temporal pole predict false memories

Chadwick

Anjum

Kumaran

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…neural similarity). To our knowledge, this approach was deployed only a few times to investigate the processing of symbolic stimuli (words), and no one investigated at the same time the organization of concepts inside and across semantic categories (Bruffaerts et al, 2013;Devereux et al, 2013). Contrary to previous studies, we estimated the similarity of our stimuli considering multiple dimensions at the same time: a low-level physical dimension (number of letters), and three semantic dimensions (a perceptual-semantic: the size of the objects referred to by the words, and two conceptual-semantic dimensions: the category and sub-category cluster).…”

Section: Multivariate Pattern Analysesmentioning

confidence: 99%

“…Researchers capitalizing from both machine learning techniques and Representational Similarity Analysis (RSA) frameworks have shown that it is possible to discriminate between words belonging to different semantic categories (e.g., animals vs tools) as well as sub-categorical clusters (e.g., mammals vs insects) using distributed patterns of brain activation (Shinkareva et al, 2011;Bruffaerts et al, 2013;Devereux et al, 2013;Fairhall and Caramazza, 2013;Simanova et al, 2014), but they did not determine if such discriminations were driven by conceptual or/and by correlated perceptual information (Naselaris and Kay, 2015). Finally, the so called "encoding" approach (modelling and predicting voxel-wise activation for different stimuli according to their defining set of features) has been successfully applied to predict brain activation during the elaboration of images and movies (Naselaris et al, 2009;Nishimoto et al, 2011), and only very recently to words (Fernandino et al, 2015a).…”

mentioning

confidence: 99%

Word meaning in the ventral visual path: a perceptual to conceptual gradient of semantic coding

et al. 2016

View full text Add to dashboard Cite

valentinaborghesani@gmail.comThe authors declare no competing financial interests. ABSTRACTThe meaning of words referring to concrete items is thought of as a multidimensional representation that includes both perceptual (e.g., average size, prototypical color) and conceptual (e.g., taxonomic class) dimensions. Are these different dimensions coded in different brain regions? In healthy human subjects, we tested the presence of a mapping between the implied real object size (a perceptual dimension) and the taxonomic categories at different levels of specificity (conceptual dimensions) of a series of words, and the patterns of brain activity recorded with functional magnetic resonance imaging in six areas along the ventral occipito-temporal cortical path. Combining multivariate pattern classification and representational similarity analysis, we found that the real object size implied by a word appears to be primarily encoded in early visual regions, while the taxonomic category and subcategorical cluster in more anterior temporal regions. This anteroposterior gradient of information content indicates that different areas along the ventral stream encode complementary dimensions of the semantic space.

show abstract

“…Other candidate regions for amodal processing could be the left or right pars triangularis and opercularis, left angular gyrus (Bonner et al, 2013), left inferior temporal gyrus (Simanova et al, 2012;Fairhall and Caramazza, 2013) or middle temporal gyrus (Badre et al, 2005), or left intraparietal sulcus (Devereux et al, 2013). …”

Section: Visual Word Processing Modelmentioning

confidence: 99%

“…An ever increasing number of studies have succeeded in detecting patterns of brain activity that reflect the identity or the semantic category of pictures (Haxby et al, 2001;Hanson et al, 2004;Shinkareva et al, 2008;Connolly et al, 2012;Mur et al, 2012;Bruffaerts et al, 2013a), written words Just et al, 2010), written words and pictures (Shinkareva et al, 2011;Simanova et al, 2012;Bruffaerts et al, 2013b;Devereux et al, 2013;Fairhall and Caramazza, 2013), written and spoken words (Akama et al, 2012;Simanova et al, 2012) as well as natural sounds (Simanova et al, 2012).…”

Section: Introductionmentioning

confidence: 99%

Left perirhinal cortex codes for similarity in meaning between written words: Comparison with auditory word input

et al. 2015

View full text Add to dashboard Cite

This is a pre-copy-editing, author-produced PDF of an article accepted for publication in Neuropsychologia following peer review. The definitive publisher-authenticated version (Liuzzi A.G., Bruffaerts R., Dupont P., Adamczuk K., Peeters R., De Deyne S., Storms G., Vandenberghe R. AbstractLeft perirhinal cortex has been previously implicated in associative coding. According to a recent experiment, the similarity of perirhinal fMRI response patterns to written concrete words is higher for words which are more similar in their meaning. If left perirhinal cortex functions as an amodal semantic hub, one would predict that this semantic similarity effect would extend to the spoken modality. We conducted an event-related fMRI experiment and evaluated whether a same semantic similarity effect could be obtained for spoken as for written words. Twenty healthy subjects performed a property verification task in either the written or the spoken modality. Words corresponded to concrete animate entities for which extensive feature generation was available from more than 1000 subjects. From these feature generation data, a concept-feature matrix was derived which formed the basis of a cosine similarity matrix between the entities reflecting their similarity in meaning (called the "semantic cossimilarity matrix"). Independently, we calculated a cosine similarity matrix between the left perirhinal fMRI activity patterns evoked by the words (called the "fMRI cossimilarity matrix"). Next, the similarity was determined between the semantic cossimilarity matrix and the fMRI cossimilarity matrix. This was done for written and spoken words pooled, for written words only, for spoken words only, as well as for crossmodal pairs. Only for written words did the fMRI cossimilarity matrix correlate with the semantic cossimilarity matrix. Contrary to our prediction, we did not find any such effect for auditory word input nor did we find cross-modal effects in perirhinal cortex between written and auditory words. Our findings situate the contribution of left perirhinal cortex to word processing at the top of the visual processing pathway, rather than at an amodal stage where visual and auditory word processing pathways have already converged.

show abstract

Representational Similarity Analysis Reveals Commonalities and Differences in the Semantic Processing of Words and Objects

Cited by 173 publications

References 57 publications

Semantic representations in the temporal pole predict false memories

Semantic representations in the temporal pole predict false memories

Word meaning in the ventral visual path: a perceptual to conceptual gradient of semantic coding

Left perirhinal cortex codes for similarity in meaning between written words: Comparison with auditory word input

Contact Info

Product

Resources

About