Neural representation of words within phrases: Temporal evolution of color-adjectives and object-nouns during simple composition

Abstract

In language, stored semantic representations of lexical items combine into an infinitude of complex expressions. While the neuroscience of composition has begun to mature, we do not yet understand how the stored representations evolve and morph during composition. New decoding techniques allow us to crack open this very hard question: we can train a model to recognize a representation in one context or time-point and assess its accuracy in another. We combined the decoding approach with magnetoencephalography recorded during a picture naming task to investigate the temporal evolution of noun and adjective representations during speech planning. We tracked semantic representations as they combined into simple two-word phrases, using single words and two-word lists as non-combinatory controls. We found that nouns were generally more decodable than adjectives, suggesting that noun representations were stronger and/or more consistent across trials than those of adjectives. When training and testing across contexts and times, the representations of isolated nouns were recoverable when those nouns were embedded in phrases, but not so if they were embedded in lists. Adjective representations did not show a similar consistency across isolated and phrasal contexts. Noun representations in phrases also sustained over time in a way that was not observed for any other pairing of word class and context. These findings offer a new window into the temporal evolution and context sensitivity of word representations during composition, revealing a clear asymmetry between adjectives and nouns. The impact of phrasal contexts on the decodability of nouns may be due to the nouns’ status as head of phrase—an intriguing hypothesis for future research.

Insights into Early Word Comprehension: Tracking the Neural Representations of Word Semantics in Infants

“…For our study, we used the vectors acquired from the skip-gram algorithm, a neural network trained to predict the context for a given word. We chose vectors from Word2Vec because the existing studies show that word vector representations of stimuli can be decoded from brain-imaging data in adults [Ruan et al, 2016, Kivisaari et al, 2019, Sudre et al, 2012, Murphy et al, 2011, Foster et al, 2021, Honari-Jahromi et al, 2021.…”

“…Previous studies conducted on brain-imaging data recorded from adults have shown success in decoding word semantics [Papadimitriou et al, 2018, Fyshe et al, 2019, Honari-Jahromi et al, 2021; we extend this idea to brain-imaging data recorded from infants. Specifically, we analyze the change in neural response patterns as the infants hear single words.…”

Multivariate pattern analysis of EEG reveals nuanced impact of negation on sentence processing in the N400 and later time windows

“…For these purposes, we leveraged the emerging multivariate pattern analysis (MVPA) approach (Grootswagers et al, 2017;King et al, 2018;. Originated from fMRI (Haxby et al, 2001), the recent application of MVPA in M/EEG research has not only advanced cognitive neuroscience research in sensory, motor, and other cognitive domains (Cichy et al, 2014;Dobs et al, 2019;Draschkow et al, 2018;, but also provided profound insights into the sentence processing literature (Fyshe, 2020;Gwilliams et al, 2022;Heikel et al, 2018;Honari-Jahromi et al, 2021;Sassenhagen and Fiebach, 2019). Briefly speaking, in the case of EEG, MVPA exploits algorithms trained on patterns of EEG signals across multiple scalp electrodes, so as to classify whether single trials belong to one of the two/more mental states (conditions).…”

Words Gold – Golden (золотой – Златой – Золотистый) in Russian Poetical Colour and Metaphoric Palette

“…If the context is not sufficient to provide information about the meaning of the adjective, a representation of a sense definition can improve the results. Honari-Jahromi, Chouinard, BlancoElorrieta, Pylkka¨nen, Fyshe (2021) focus their attention on the collocations with adjectives as well and use a decoding approach to address the general question of how combinatory contexts affect the neural representations of word meanings. Their combinatory contexts were all noun phrases comprised of a color adjective and an object-describing noun, such as 'blue cup.'…”