The overall effect of hand gesture on learning of segmental phonology is limited. Implications for theories of hand gesture are discussed in terms of the role it plays at different linguistic levels.

Co-speech hand gestures are a type of multimodal input that has received relatively little attention in the context of second language learning. The present study explored the role that observing and producing different types of gestures plays in learning novel speech sounds and word meanings in an L2. Naïve English-speakers were taught two components of Japanese—novel phonemic vowel length contrasts and vocabulary items comprised of those contrasts—in one of four different gesture conditions: Syllable Observe, Syllable Produce, Mora Observe, and Mora Produce. Half of the gestures conveyed intuitive information about syllable structure, and the other half, unintuitive information about Japanese mora structure. Within each Syllable and Mora condition, half of the participants only observed the gestures that accompanied speech during training, and the other half also produced the gestures that they observed along with the speech. The main finding was that participants across all four conditions had similar outcomes in two different types of auditory identification tasks and a vocabulary test. The results suggest that hand gestures may not be well suited for learning novel phonetic distinctions at the syllable level within a word, and thus, gesture-speech integration may break down at the lowest levels of language processing and learning.

This study examined whether auditory training coupled with hand gesture can improve non-native speakers’ auditory learning of phonemic vowel length contrasts in Japanese. Hirata and Kelly (2010) found that observing hand gesture that moved along with the rhythm of spoken short and long vowels in Japanese did not uniquely contribute to non-native speakers’ auditory learning. The present study compared effects of four types of training to examine whether there is a more effective method: (1) producing syllabic-rhythm gesture, (2) observing syllabic-rhythm gesture, (3) producing moraic-rhythm gesture, and (4) observing moraic-rhythm gesture. Each of native English speakers (N = 88) participated in one of the four types of training in four sessions, and took a pretest and a posttest that measured their ability to auditorily identify the vowel length of novel words without hand gesture. Tested disyllable pairs had the contrast in the first and the second syllables, spoken in sentences at slow and fast rates. Results showed that all four groups improved significantly (9%), but the amount of improvement did not differ. However, ‘observing syllabic-rhythm gesture’ was the only condition in which auditory learning was balanced between the first and the second syllable contexts and between the slow and fast rates.