No abstract

Eye-tracking findings suggest people prefer to ground their spoken language comprehension by focusing on recently seen events more than anticipating future events: When the verb in NP1-VERB-ADV-NP2 sentences was referentially ambiguous between a recently depicted and an equally plausible future clipart action, listeners fixated the target of the recent action more often at the verb than the object that hadn’t yet been acted upon. We examined whether this inspection preference generalizes to real-world events, and whether it is (vs. isn’t) modulated by how often people see recent and future events acted out. In a first eye-tracking study, the experimenter performed an action (e.g., sugaring pancakes), and then a spoken sentence either referred to that action or to an equally plausible future action (e.g., sugaring strawberries). At the verb, people more often inspected the pancakes (the recent target) than the strawberries (the future target), thus replicating the recent-event preference with these real-world actions. Adverb tense, indicating a future versus past event, had no effect on participants’ visual attention. In a second study we increased the frequency of future actions such that participants saw 50/50 future and recent actions. During the verb people mostly inspected the recent action target, but subsequently they began to rely on tense, and anticipated the future target more often for future than past tense adverbs. A corpus study showed that the verbs and adverbs indicating past versus future actions were equally frequent, suggesting long-term frequency biases did not cause the recent-event preference. Thus, (a) recent real-world actions can rapidly influence comprehension (as indexed by eye gaze to objects), and (b) people prefer to first inspect a recent action target (vs. an object that will soon be acted upon), even when past and future actions occur with equal frequency. A simple frequency-of-experience account cannot accommodate these findings.

This research report examines the occurrence of listener visual cues during nonunderstanding episodes and investigates raters’ sensitivity to those cues. Nonunderstanding episodes (n = 21) and length-matched understanding episodes (n = 21) were taken from a larger dataset of video-recorded conversations between second language (L2) English speakers and a bilingual French-English interlocutor (McDonough, Trofimovich, Dao, & Abashidze, 2018). Episode videos were analyzed for the occurrence of listener visual cues, such as head nods, blinks, facial expressions, and holds. Videos of the listener’s face were manipulated to create three rating conditions: clear voice/clear face, distorted voice/clear face, and clear voice/blurred face. Raters in the same speech community (N = 66) were assigned to a video condition to assess the listener’s comprehension. Results revealed differences in the occurrence of listener visual cues between the understanding and nonunderstanding episodes. In addition, raters gave lower ratings of listener comprehension when they had access to the listener’s visual cues.

To confirm the role of social factors in mediating cognitive processes, this systematic replication study seeks to extend the generalizability of an exploratory study (McDonough, Crowther, Kielstra & Trofimovich 2015) that reported a positive association between eye gaze and second language (L2) speakers’ responses to recasts. For this replication, L2 English speakers (N = 74) carried out communicative tasks with research assistants who provided recasts in response to non-targetlike forms while both interlocutors’ eye gaze behavior was tracked. Transcripts were analyzed for the occurrence of recasts in response to different error types, recast length, and L2 speaker responses. Eye gaze length for the research assistants (RAs) when producing the recast move and the L2 speaker when responding to the recast were obtained in seconds, and mutual gaze (i.e., simultaneous looking) was included as a binary eye gaze variable. A logistic regression model confirmed the findings of McDonough et al. (2015), with both L2 speaker and mutual eye gaze predictive of targetlike responses; however, the effect of L2 speaker's eye gaze duration was in the opposite direction as compared to the initial study. The implications are discussed in terms of understanding the role of eye gaze in face to face interaction.

In interpreting spoken sentences in event contexts, comprehenders both integrate their current interpretation of language with the recent past (e.g., events they have witnessed) and develop expectations about future event possibilities. Tense cues can disambiguate this linking but temporary ambiguity in their interpretation may lead comprehenders to also rely on further, situation-specific cues (e.g., an actor's gaze as a cue to his future actions). How comprehenders reconcile these different cues in real time is an open issue that we must address to accommodate comprehension. It has been suggested that relating a referential expression (e.g., a verb) to a referent (e.g., a recent event) is preferred over relying on other cues that refer to the future and are not yet referentially grounded (“recent-event preference”). Two visual-world eye-tracking experiments compared this recent-event preference with effects of an actor's gaze and of tense/temporal adverbs as cues to a future action event. The results revealed that people overall preferred to focus on the recent (vs. future) event target in their interpretation, suggesting that while a congruent and incongruent actor gaze can jointly with futuric linguistic cues neutralize the recent-event preference late in the sentence, the latter still plays a key role in shaping participants' initial verb-based event interpretation. Additional post-experimental memory tests provided insight into the longevity of the gaze effects.

Visual cues may help second language (L2) speakers perceive interactional feedback and reformulate their nontarget forms, particularly when paired with recasts, as recasts can be difficult to perceive as corrective. This study explores whether recasts have a visual signature and whether raters can perceive a recast’s corrective function. Transcripts of conversations between a bilingual French–English interlocutor and L2 English university students ( n = 24) were analysed for recasts and noncorrective repetitions with rising and declarative intonation. Videos of those excerpts ( k = 96) were then analysed for the interlocutor’s provision of visual cues during the recast and repetition turns, including eye gaze duration, nods, blinks, and other facial expressions (frowns, eyebrow raises). The videos were rated by 96 undergraduate university students who were randomly assigned to three viewing conditions: clear voice/clear face, clear voice/blurred face, or distorted voice/clear face. Using a 100-millimeter scale with two anchor points (0% = he’s making a comment, and 100% = he’s correcting an error), they rated the corrective function of the interlocutors’ responses while their eye gaze was tracked. Raters reliably distinguished recasts from repetitions through their ratings (although they were generally low), but not through their eye gaze behaviors.

Aims and objectives: The current study examines the role of grammatical and positional parallelism on ambiguous German pronoun resolution in monolingual speakers and highly proficient L2 speakers with L1 Georgian. In particular, the study asks whether an object pronoun ihn in a sentence initial position could refer to the non-subject antecedent in a preceding subject–verb–object sentence. Furthermore, it investigates whether L1 and L2 speakers show similar preferences when resolving ambiguous object pronouns. Methodology: Two visual world eye-tracking and two offline experiments were conducted with L1 and L2 speakers of German. During the eye-tracking session, the task was simply to look and listen, while in the offline test, participants were required to decide on the assumed referent of the ambiguous pronoun. Data and analysis: Linear-mixed effect models were applied to the eye-movement data. Two-sample t tests were used in the analysis of the offline data. Findings and conclusions: Eye-movement results revealed a bias toward the subject antecedent in L1 speakers, while the L2 speakers showed a non-subject preference when resolving the referent of the object pronoun. The offline experiments supported the eye-tracking findings of both groups, particularly as regards the role of grammatical parallelism in the resolution of ambiguous pronouns by the L2 speakers. These findings are interpreted in the context of language-specific processing strategies in anaphora resolution. It appears that L1 and L2 speakers may be using different information-structural cues to resolve ambiguous pronouns. Concretely, a less strongly internalized knowledge of the target language or the influence of the first language may produce alternative processing mechanisms in L2 speakers. Originality: This study fills the research gap on ambiguous object pronoun resolution of L1 and L2 speakers of German. Implications: The findings of this study add additional information to the growing body of research into pronoun resolution. It focuses on the resolution of the German object pronoun within an information-structural framework of language processing in L1 and L2.

The present work is a description and an assessment of a methodology designed to quantify different aspects of the interaction between language processing and the perception of the visual world. The recording of eye-gaze patterns has provided good evidence for the contribution of both the visual context and linguistic/world knowledge to language comprehension. Initial research assessed object-context effects to test theories of modularity in language processing. In the introduction, we describe how subsequent investigations have taken the role of the wider visual context in language processing as a research topic in its own right, asking questions such as how our visual perception of events and of speakers contributes to comprehension informed by comprehenders' experience. Among the examined aspects of the visual context are actions, events, a speaker's gaze, and emotional facial expressions, as well as spatial object configurations. Following an overview of the eyetracking method and its different applications, we list the key steps of the methodology in the protocol, illustrating how to successfully use it to study visually-situated language comprehension. A final section presents three sets of representative results and illustrates the benefits and limitations of eye tracking for investigating the interplay between the perception of the visual world and language comprehension. Video Link The video component of this article can be found at https://www.jove.com/video/57694/ 2). In this 'visual world' eye-tracking version, the inspection of objects is guided by language. When the comprehenders hear the zebra, for instance, their inspection of a zebra on the screen is taken to reflect that they are thinking about the animal. In what is known as the visual world paradigm, a comprehender's eye gaze is taken to reflect spoken language comprehension and the activation of associated knowledge (e.g., listeners also inspect the zebra when they hear grazing, indicating an action performed by the zebra) 2. Such inspections suggest a systematic link between language-world relations and eye movements 2. A common way to quantify this link is by computing the proportion of looks to different predetermined regions on a screen. This allows researchers to directly compare (across conditions, by participants and items) the amount of attention given to different objects at a particular time and how these values change in millisecond resolution.