Attention to Speech-Accompanying Gestures: Eye Movements and Information Uptake

Abstract

There is growing evidence that addressees in interaction integrate the semantic information conveyed by speakers' gestures. Little is known, however, about whether and how addressees' attention to gestures and the integration of gestural information can be modulated. This study examines the influence of a social factor (speakers' gaze to their own gestures), and two physical factors (the gesture's location in gesture space and gestural holds) on addressees' overt visual attention to gestures (direct fixations of gestures) and their uptake of gestural information. It also examines the relationship between gaze and uptake. The results indicate that addressees' overt visual attention to gestures is affected both by speakers' gaze and holds but for different reasons, whereas location in space plays no role. Addressees' uptake of gesture information is only influenced by speakers' gaze. There is little evidence of a direct relationship between addressees' direct fixations of gestures and their uptake.

Perception of co-speech gestures in aphasic patients: A visual exploration study during the observation of dyadic conversations

“…Previous studies analysed healthy subjects' visual exploration of co-speech gestures while observing an actor who was retelling cartoon stories. These studies found that gestures attract only a minor portion of attention (2e7%), while the speaker's face is much more fixated (90e95%) (Beattie, Webster, & Ross, 2010;Gullberg & Holmqvist, 1999, 2006Gullberg & Kita, 2009;Nobe, Hayamizu, Hasegawa, & Takahashi, 2000). To the best of our knowledge the visual exploration behaviour of co-speech gestures has not been studied in aphasic patients.…”

How to Attract Students’ Visual Attention

“…It is possible for a student to focus on a visual target (teacher) without paying attention to it (i.e. 'looking without observing') and, conversely, paying attention to something without directly focusing on it ('observing without looking') [18]. There is a very crucial and subtle difference between the two.…”

What's Left Unsaid: How Nonverbal Influence Compares with Verbal Influence

“…Although this may serve as an explanation for how accuracy rates were similar across conditions, it is important to note that the mechanisms behind gestural misinformation are not comparable with those of verbal misinformation. If nonverbal misinformation is incorporated into an eyewitness' representation, it occurs through a more covert process: gestures are not readily noticed in conversation (Gullberg & Holmqvist, 2002, 2006, even when fixated (Gullberg & Kita, 2009), and listeners often cannot identify gesture as the source of information retrospectively (Kelly et al, 1999). This is an important consideration to take into account because the success of gestures in manipulating memory reports could be attributed to a lack of source monitoring.…”

“…Although the effects of verbal misinformation are mediated by source monitoring, gestures may not be subject to the same credibility assessments, as listeners do not subscribe to information conveyed by them overtly. Gestures are not often attended to in conversation (Gullberg & Holmqvist, 2002, 2006Gullberg & Kita, 2009) and listeners show little awareness of when information has been conveyed to them nonverbally: Kelly et al (1999) report that when presenting the sentence "my brother went to the gym" with a "shooting a basketball" gesture, listeners not only extracted the critical "basketball" information from gesture, but often remembered this as being part of speech. Thus, while gestures can convey information that carries the same semantic value as information in speech, they often provide listeners with this information without any trace of having done so.…”