Understanding gesture: Is the listener’s motor system involved?

Abstract

Listeners are able to glean information from the gestures that speakers produce, seemingly without conscious awareness. However, little is known about the mechanisms that underlie this process.Research on human action understanding shows that perceiving another's actions results in automatic activation of the motor system in the observer, which then affects the observer's understanding of the actor's goals. We ask here whether perceiving another's gesture can similarly result in automatic activation of the motor system in the observer. In Experiment 1, we first establish a new procedure that uses listener response times to study how gesture impacts sentence comprehension. In Experiment 2, we use this procedure, in conjunction with a secondary motor task, to investigate whether the listener's motor system is involved in this process. We show that moving arms and hands (but not legs and feet) interferes with the listener's ability to use information conveyed in a speaker's hand gestures. Our data thus suggest that understanding gesture relies, at least in part, on the listener's own motor system.When we watch others act on the world, our own motor systems are activated, which, in turn, affects how we interpret the actors' goals (e.g., Buccino et al., 2001;Hamilton, Wolpert & Frith, 2004;Wilson, Collins & Bingham, 2005;Sebanz, Bekkering & Knoblich, 2006). Here we ask whether this same process takes place even when the actions we observe do not have a direct effect on the world, but rather impact the world indirectly through their communicative potential. When people speak, they often move their hands--they gesture--and listeners are able to glean substantive information from these gestures, although typically without being aware of doing so. Does watching a speaker who gestures activate our own motor system? Perception and action have been shown to be linked, both within an individual and across individuals. In one line of studies, participants must perceive and/or produce bilateral movements that are perceptually and motorically difficult. Without training, humans can easily perceive and produce movements that are either identical (e.g., simultaneously moving both pointer fingers left and right together-a 0-degree phase) or symmetric (e.g., simultaneously moving both pointer fingers in and out together-a 180-degree phase). Distinguishing or producing bilateral movements at any other phase is difficult, requires extensive practice, and falls apart once movements reach a threshold frequency. Within an individual, learning to perceive distinctions at difficult phases (perception) improves the ability to produce movements (action) at those phases-movements that otherwise would require extensive motor practice to master (Wilson, Snapp-Childs & Bingham, 2010; see also Bingham, Schmidt & Zaal, 1999;Zaal, Bingham & Schmidt, 2000). Conversely, learning to produce movements at particular phases, without visual feedback of one's own body, improves perceptual discrimination of those phases specifically (Hecht, Vogt & ...

Sensitivity of alpha and beta oscillations to sensorimotor characteristics of action: An EEG study of action production and gesture observation

“…The similarities we found in EEG patterns for action production and gesture observation support the notion that our own sensorimotor systems are active when we observe other people gesture (see also Ping, Goldin-Meadow & Beilock, under review). The topographic differences in sensitivity to weight between gesture observation and action execution are consistent with work suggesting that information related to self and other is represented differently in the brain (Walla, Greiner, Duregger, Deecke, & Thurner, 2007).…”

Gesture as representational action: A paper about function

“…Seeing gesture might also support learning through the same mechanisms as producing gesture, that is, by engaging the motor system. Listeners recruit their own motor systems when listening to speakers who gesture (Ping, Goldin-Meadow, & Beilock, 2014), and neuroimaging research suggests that recruiting the motor system may be key in learning. Adults learn more foreign words if they are taught those words while seeing someone produce meaningful iconic gestures, compared to seeing someone produce meaningless movements (Macedonia, Muller, & Friederici, 2011).…”

The comprehensibility of pantomimes produced by people with aphasia

“…) and co‐speech gestures can be comprehensible (Beattie and Shovelton , Kelly and Church , Ping et al . ), but very little is known about how much and what type of information these gestures convey. As there are no clear conventions on the meaning of pantomime, its meaning is probably often ambiguous and context dependent.…”