Identifying thematic roles from neural representations measured by functional magnetic resonance imaging

Wang, Jing; Cherkassky, Vladimir L.; Yang, Ying; Chang, Kai-Min; Vargas, Robert; Diana, Nicholas; Just, Marcel Adam

doi:10.1080/02643294.2016.1182480

Cited by 27 publications

(22 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dog in Dog chases cat is neurally distinguishable from the dog in Cat chases dog , indicating that the neural encoding is more than just the sum of the individual concept representations. In another study using the same approach as the current one, a classifier was able to reliably discriminate Monkey pats rabbit and Rabbit pats monkey and to correctly assign thematic roles to the two animal concepts (Wang, J., Cherkassky, V. L., Just, 2016). Thus, the thematic role encoding could be part of a structured sentence representation that plays a role in determining how the components of a concept are selectively modulated.…”

Section: Discussionmentioning

confidence: 78%

“…Other studies have found distinct activation patterns associated with the neural representations of concepts of varying degrees of semantic abstractness (Anderson et al, 2014; Ghio et al, 2016; Wang et al, 2013). A few studies have further demonstrated the ability to associate brain activation patterns with inter-concept relations in a proposition (Frankland and Greene, 2015; Wang et al, 2016). However, characterizing the neural representations of sentences and the effect of contexts has remained a considerable challenge.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neural representations of the concepts in simple sentences: Concept activation prediction and context effects

2017

Self Cite

View full text Add to dashboard Cite

Although it has been possible to identify individual concepts from a concept’s brain activation pattern, there have been significant obstacles to identifying a proposition from its fMRI signature. Here we demonstrate the ability to decode individual prototype sentences from readers’ brain activation patterns, by using theory-driven regions of interest and semantic properties. It is possible to predict the fMRI brain activation patterns evoked by propositions and words which are entirely new to the model with reliably above-chance rank accuracy. The two core components implemented in the model that reflect the theory were the choice of intermediate semantic features and the brain regions associated with the neurosemantic dimensions. This approach also predicts the neural representation of object nouns across participants, studies, and sentence contexts. Moreover, we find that the neural representation of an agent-verb-object proto-sentence is more accurately characterized by the neural signatures of its components as they occur in a similar context than by the neural signatures of these components as they occur in isolation.

show abstract

Section: Discussionmentioning

confidence: 78%

Section: Introductionmentioning

confidence: 99%

Neural representations of the concepts in simple sentences: Concept activation prediction and context effects

2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…Greene (2015, 2020) used sentence stimuli to isolate distinct areas in left superior temporal sulcus (STS) that are sensitive to the identity of the agent vs. the patient. Wang et al (2016) found that the same (or nearby) STS regions also contained information about thematic roles in videos depicting agentpatient interactions. However, the latter study identified a number of other regions that were sensitive to thematic role information, including clusters in the right posterior middle temporal gyrus and right angular gyrus, suggesting that left STS is not the only region implicated in thematic role processing.…”

Section: Relationship To Theories Of Semantics In the Brainmentioning

confidence: 92%

The language network is recruited but not required for non-verbal event semantics

Ivanova

Mineroff

Zimmerer

et al. 2019

Preprint

View full text Add to dashboard Cite

Consistent with longstanding findings from neuropsychology, several brain regions in left frontal and temporal cortex respond robustly and selectively to language [1]. These regions, often referred to as the "language network", respond more strongly to meaningful stimuli (like words and sentences) than to stimuli devoid of meaning (like pseudowords and Jabberwocky sentences) [2]. But are these regions selectively recruited in processing linguistic meaning? Or do they instead store and/or process complex semantic information independent of its format (sentences or pictures)? In Experiment 1, we scanned participants with fMRI while they performed a semantic plausibility judgment task vs. a difficult perceptual control task on sentences and line drawings that describe/depict simple agent-patient interactions. We found that the language regions responded more strongly when participants performed the semantic task compared to the perceptual task, for both sentences and pictures (although sentences elicited overall stronger responses). Thus, healthy adults engage language regions when processing non-verbal meanings. But is this engagement necessary for understanding pictorial depictions of events? In Experiment 2, we tested two individuals with global aphasia, who have sustained massive damage to perisylvian language areas and display severe language difficulties, together with a group of control participants. Individuals with aphasia were at chance on a task of matching the sentences and pictures. However, they performed close to controls in assessing the plausibility of pictorial depictions of agent-patient interactions. Taken together, these results indicate that the left fronto-temporal language system is recruited but not necessary for processing complex non-verbal meanings.

show abstract

“…First, the relatively uncontrolled properties of the naturalistic paradigms and the substantial differences across stimuli in both the localizer and main tasks might have been unsuitable for detecting subtler linguistic distinctions; and the definition of a single participant-specific fROI in each mask might have compromised our ability to identify distinctions among small regions that lie in close proximity to one another (e.g., Humphries et al, 2005;Hagoort, 2014;Wilson et al, 2018). Nonetheless, whether such distinctions as previously reported in the literature are replicable, and whether they reflect language-specific functions rather than more general, conceptual processes, remains debated (For one such example, see Dapretto and Bookheimer, 1999;Siegelman et al, 2019; for another, see Frankland and Greene, 2015;Wang et al, 2016;Anderson et al, 2018; see also Vigliocco et al, 2011;Moseley and Pulvermüller, 2014).…”

Section: Why Functional Dissociations Among Language Regions Might Gomentioning

confidence: 99%

No evidence for functional distinctions across fronto-temporal language regions in their temporal receptive windows

Blank

Fedorenko

2019

Preprint

View full text Add to dashboard Cite

words)The "core language network" consists of left temporal and frontal regions that are selectively engaged in linguistic processing. Whereas the functional differences across these regions have long been debated, many accounts propose distinctions in terms of representational grain-size-e.g., words vs. sentences-or processing time-scale, i.e., operating on local linguistic features vs. larger spans of input. Indeed, the topography of language regions appears to overlap with a cortical hierarchy reported by Lerner et al. (2011) wherein mid-posterior temporal regions are sensitive to low-level features of speech, surrounding areas-to word-level information, and inferior frontal areas-to sentence-level information and beyond. However, the correspondence between the language network and this hierarchy of "temporal receptive windows" (TRWs) is difficult to establish because the precise anatomical locations of language regions vary across individuals. To directly test this correspondence, we first identified language regions in each participant with a task-based localizer, which confers high functional resolution to the study of TRWs (traditionally based on stereotactic coordinates); then, we characterized regional TRWs with the naturalistic story listening paradigm of Lerner et al. (2011), which augments task-based characterizations of the language network by more closely resembling comprehension "in the wild". We find no region-by-TRW interactions across temporal and inferior frontal regions, which are all sensitive to both word-level and sentence-level information. Therefore, the language network as a whole constitutes a unique stage of information integration within a broader cortical hierarchy. Highlights:• Language regions are identified with task-based, participant-specific localization.• A progressively scrambled naturalistic story probes regional processing timescales.• Widespread sensitivity to scrambling at the timescales of both words and sentences.• No processing timescale distinctions across temporal and inferior-frontal regions.• These regions all occupy a common, unique stage in a broader processing hierarchy.reliably track any locally well-formed input even in the face of coarser, global disorder (morphemes can be extracted even from ungrammatical sequences of unrelated words); but a region with a longer integration timescale (e.g., on the order of phrases) could not reliably track such locally-intact-yet-globally-incoherent input (phrases would be difficult to establish in such sequences). Therefore, a straightforward prediction that follows from the "hierarchy of processing timescales" hypothesis is that different language regions should exhibit distinct patterns of input tracking when well-formedness deteriorates from coarser, more global disruptions to finer, more local violations.Indeed, such a response profile consistent with a hierarchy of integration timescales has been reported in a set of left temporal and frontal areas, whose topography appears consistent with the core language network (Le...

show abstract

Identifying thematic roles from neural representations measured by functional magnetic resonance imaging

Cited by 27 publications

References 25 publications

Neural representations of the concepts in simple sentences: Concept activation prediction and context effects

Neural representations of the concepts in simple sentences: Concept activation prediction and context effects

The language network is recruited but not required for non-verbal event semantics

No evidence for functional distinctions across fronto-temporal language regions in their temporal receptive windows

Contact Info

Product

Resources

About