Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic Contrasts

Abstract

It is well established that hand gestures affect comprehension and learning of semantic aspects of a foreign language (FL). However, much less is known about the role of hand gestures in lower-level language processes, such as perception of phonemes. To address this gap, we explored the role that metaphoric gestures play in perceiving FL speech sounds that varied on two dimensions: length and intonation. English speaking adults listened to Japanese length contrasts and sentence-final intonational distinctions in the context of congruent, incongruent and no gestures. For intonational contrasts, identification was more accurate for congruent gestures and less accurate for incongruent gestures relative to the baseline no gesture condition. However, for the length contrasts, there was no such clear and consistent pattern, and in fact, congruent gestures made speech processing more effortful. We conclude that metaphoric gestures help with some-but not all-novel speech sounds in a FL, suggesting that gesture and speech are phonemically integrated to differing extents depending on the nature of the gesture and/or speech sound.

Do gestures during training facilitate L2 lexical stress acquisition by Dutch learners of Spanish?

“…Although the descriptives support the direction of our hypotheses, the multinomial logistic regression analysis reveals that the difference between the three gesture conditions is not significant for the outcome in which learning takes place, which is of most interest for this study. This is in line with those prior studies that did not find an effect of gestural training in the context of L2 (supra)segment acquisition [14,15,16]. The absence of a significant difference between gesture conditions might be explained in several ways.…”

“…This might be surprising, considering the fact that a visual cue such as a written accent explicitly guides the learner as to which syllable is to be emphasised within the word. However, it does corroborate prior research which shows that the specific context in which L2 phonology acquisition does or does not occur matters when it comes to the facilitating role of gestures [12,15,28]. In addition, the use of cognates, while alleviating semantic processing load, does result in the use of less balanced item lists.…”

“…As accurate segment and prosody production contributes to L2 learners' intelligibility, comprehensibility and accentedness [9,10,11], research on the interplay between gestural and verbal input in this domain is especially relevant. However, regarding the benefits of gestures in the acquisition of L2 sounds, prior work presents contrasting results: Some studies find that gestures seen or produced during training facilitate the learning of both L2 segments and prosodic features [e.g., 12,13], others find no effect or report contexts in which gestures might even be unhelpful [14,15,16]. One possible reason for these contradicting results may be the type of gesture that is used during training.…”

Observing and Producing Pitch Gestures Facilitates the Learning of Mandarin Chinese Tones and Words

“…The results of the experiment showed that there was no noticeable benefit for participants when they learned vowel length by viewing videos showing length gestures as opposed to viewing videos that did not show such gestures. More recently, Kelly et al (2017) suggested that it may be possible to safely narrow down the effective use of perhaps the utility of visuospatial gestures in pronunciation learning is limited to the use of pitch gestures for the learning of intonation patterns (but not thus excluding the use of various types of metaphoric gestures for the study of duration). There might be a set of several possible reasons that can explain for the discrepancy between the results of the previously mentioned studies.…”

“…We would like to suggest that the type of metaphorical gestures used by Kelly, Hirata, and colleagues (Hirata & Kelly, 2010;Hirata et al, 2014;Kelly & Lee, 2012;Kelly et al, 2017) may have had an influence too. Specifically, the mora gestures used in the studies of Kelly, Hirata, and colleagues (e.g., the short vertical chopping movements) might have come across as "nonintuitive" to English speakers and thus did facilitate (or even hindered) their learning of durational information in the second language (see also the comments on the lack of effectiveness of length gestures in Kelly et al, 2017). The fact that other studies like Gluhareva and Prieto (2017) have found that observing other types of rhythmic gestures (e.g., beat gestures) has a positive effect on general pronunciation results leads us to suspect that perhaps the pitch gestures must seem natural to have positive results.…”