Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture‐sentence verification

Knoeferle, Pia; Urbach, Thomas P.; Kutas, Marta

doi:10.1111/j.1469-8986.2010.01080.x

Cited by 53 publications

(65 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have reported reliable correlations between N400 congruence effects at the verb and end-of-sentence congruence response times in young adults (Knoeferle et al, 2011). Participants with a small N400 congruence effect at the verb tended to exhibit a large response time congruence effect at sentence end, and vice versa.…”

Section: Verb-action Versus Thematic Role Relations Mismatches: Erps mentioning

confidence: 97%

“…Depending on when they assign the thematic (agent) role to the first noun phrase, this may occur as soon as the first noun, or perhaps not until the verb. If both the role relations and verb action mismatch effects appear at the verb, they may be indexed by larger negative mean amplitude ERPs compared with matches (N400) as reported for active sentences (Knoeferle et al, 2011;Wassenaar & Hagoort, 2007).…”

Section: Verb-action Versus Thematic Role Relations Mismatches: Erps mentioning

confidence: 99%

“…Knoeferle et al (2011), for instance, examined verb-action relationships as participants read a subject-verb-object sentence and verified whether or not the verb matched an immediately preceding depicted action. Two qualitatively distinct ERP effects emerged (one at the verb, the other at the post-verbal object noun)…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Different mechanisms for role relations versus verb–action congruence effects: Evidence from ERPs in picture–sentence verification

2014

Self Cite

View full text Add to dashboard Cite

Extant accounts of visually situated language processing do make general predictions about visual context effects on incremental sentence comprehension; these, however, are not sufficiently detailed to accommodate potentially different visual context effects (such as a scene-sentence mismatch based on actions versus thematic role relations, e.g., Altmann & Kamide, 2007;Knoeferle & Crocker, 2007;Taylor & Zwaan, 2008;Zwaan & Radvansky, 1998).To provide additional data for theory testing and development, we collected event-related brain potentials (ERPs) as participants read a subject-verbobject sentence (500 ms SOA in Experiment 1 and 300 ms SOA in Experiment 2), and post-sentence verification times indicating whether or not the 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 verb and/or the thematic role relations matched a preceding picture (depicting two participants engaged in an action). Though incrementally processed, these two types of mismatch yielded different ERP effects. Role-relation mismatch effects emerged at the subject noun as anterior negativities to the mismatching noun, preceding action mismatch effects manifest as centro-parietal N400s greater to the mismatching verb, regardless of SOAs. These two types of mismatch manipulations also yielded different effects post-verbally, correlated differently with a participant's mean accuracy, verbal working memory and visual-spatial scores, and differed in their interactions with SOA. Taken together these results clearly implicate more than a single mismatch mechanism for extant accounts of picture-sentence processing to accommodate.

show abstract

Section: Verb-action Versus Thematic Role Relations Mismatches: Erps mentioning

confidence: 97%

Section: Verb-action Versus Thematic Role Relations Mismatches: Erps mentioning

confidence: 99%

See 1 more Smart Citation

Different mechanisms for role relations versus verb–action congruence effects: Evidence from ERPs in picture–sentence verification

2014

Self Cite

View full text Add to dashboard Cite

show abstract

“…If that is true, however, then it is unclear why picture-sentence incongruence processing does not always elicit a P600. When participants inspected scenes depicting an agent-action-patient event and subsequently read a related sentence in which the verb either matched or mismatched the previously inspected action, no P600 differences emerged (Knoeferle et al, 2011b). Instead, mean amplitude N400s to the verb were more negative and post-sentence verification response times were longer for verb-action mismatches than matches.…”

Section: Ambiguity In Linking Of Erp Effects To Comprehension Sub-promentioning

confidence: 99%

Language comprehension in rich non-linguistic contexts: combining eye-tracking and event-related brain potentials

Knoeferle

2015

Cognitive Neuroscience of Natural Language Use

Self Cite

View full text Add to dashboard Cite

The present chapter reviews the literature on visually situated language comprehension against the background that most theories of real-time sentence comprehension have ignored rich non-linguistic contexts. However, listeners' eye movements to objects during spoken language comprehension, as well as their eventrelated brain potentials (ERPs) have revealed that non-linguistic cues play an important role for real-time comprehension. In fact, referential processes are rapid and central in visually situated spoken language comprehension and even abstract words are rapidly grounded in objects through semantic associations. Similar ERP responses for non-linguistic and linguistic effects on comprehension suggest these two information sources are on a par in informing language comprehension. ERPs further revealed that non-linguistic cues affect lexical-semantic as well as compositional processes, thus further cementing the role of rich non-linguistic context in language comprehension. However, there is also considerable ambiguity in the linking between comprehension processes and each of these two measures (eye movements and ERPs). Combining eye-tracking and event-related brain potentials would improve the interpretation of individual measures and thus insights into visually-situated language comprehension.Knoeferle, P. (to appear). Language comprehension in rich non-linguistic contexts: combining eye tracking and event-related brain potentials. In: Roel, Williams (Ed.). Towards a cognitive neuroscience of natural language use. Cambridge: Cambridge University Press.

show abstract

“…The photographs were the grayscale versions of the ones used in the production trials. This paradigm has been used extensively, and a recent study supports its suitability for studying online situated language comprehension (Knoeferle, Urbach, & Kutas, 2011). By choosing situated paradigms for both production and comprehension trials, we maximize comparability and ensure that the difference between the two only lies in language processing.…”

Section: Comprehensionmentioning

confidence: 99%

Syntactic priming and the lexical boost effect during sentence production and sentence comprehension: An fMRI study

et al. 2013

View full text Add to dashboard Cite

a b s t r a c tBehavioral syntactic priming effects during sentence comprehension are typically observed only if both the syntactic structure and lexical head are repeated. In contrast, during production syntactic priming occurs with structure repetition alone, but the effect is boosted by repetition of the lexical head. We used fMRI to investigate the neuronal correlates of syntactic priming and lexical boost effects during sentence production and comprehension. The critical measure was the magnitude of fMRI adaptation to repetition of sentences in active or passive voice, with or without verb repetition. In conditions with repeated verbs, we observed adaptation to structure repetition in the left IFG and MTG, for active and passive voice. However, in the absence of repeated verbs, adaptation occurred only for passive sentences. None of the fMRI adaptation effects yielded differential effects for production versus comprehension, suggesting that sentence comprehension and production are subserved by the same neuronal infrastructure for syntactic processing.

show abstract

Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture‐sentence verification

Cited by 53 publications

References 51 publications

Different mechanisms for role relations versus verb–action congruence effects: Evidence from ERPs in picture–sentence verification

Different mechanisms for role relations versus verb–action congruence effects: Evidence from ERPs in picture–sentence verification

Language comprehension in rich non-linguistic contexts: combining eye-tracking and event-related brain potentials

Syntactic priming and the lexical boost effect during sentence production and sentence comprehension: An fMRI study

Contact Info

Product

Resources

About