Bilinguals Show Proportionally Greater Benefit From Visual Speech Cues and Sentence Context in Their Second Compared to Their First Language

Abstract

Objectives: Speech perception in noise is challenging, but evidence suggests that it may be facilitated by visual speech cues (e.g., lip movements) and supportive sentence context in native speakers. Comparatively few studies have investigated speech perception in noise in bilinguals, and little is known about the impact of visual speech cues and supportive sentence context in a first language compared to a second language within the same individual. The current study addresses this gap by directly investigating the extent to which bilinguals benefit from visual speech cues and supportive sentence context under similarly noisy conditions in their first and second language.Design: Thirty young adult English-French/French-English bilinguals were recruited from the undergraduate psychology program at Concordia University and from the Montreal community. They completed a speech perception in noise task during which they were presented with video-recorded sentences and instructed to repeat the last word of each sentence out loud. Sentences were presented in three different modalities: visual-only, auditory-only, and audiovisual. Additionally, sentences had one of two levels of context: moderate (e.g., "In the woods, the hiker saw a bear.") and low (e.g., "I had not thought about that bear."). Each participant completed this task in both their first and second language; crucially, the level of background noise was calibrated individually for each participant and was the same throughout the first language and second language (L2) portions of the experimental task.Results: Overall, speech perception in noise was more accurate in bilinguals' first language compared to the second. However, participants benefited from visual speech cues and supportive sentence context to a proportionally greater extent in their second language compared to their first. At the individual level, performance during the speech perception in noise task was related to aspects of bilinguals' experience in their second language (i.e., age of acquisition, relative balance between the first and the second language). Conclusions: Bilinguals benefit from visual speech cues and sentence context in their second language during speech in noise and do so to a greater extent than in their first language given the same level of background noise. Together, this indicates that L2 speech perception can be conceptualized within an inverse effectiveness hypothesis framework with a complex interplay of sensory factors (i.e., the quality of the auditory speech signal and visual speech cues) and linguistic factors (i.e., presence or absence of supportive context and L2 experience of the listener).

Audiovisual speech perception in noise in younger and older bilinguals.

“…Furthermore, these results add to a growing literature suggesting that bilinguals benefit from visual speech cues (e.g., Chauvin & Phillips, 2022) and sentence context (e.g., Kousaie et al, 2019) in their L2. Additionally, the current results extend this further to suggest that both younger and older bilinguals can benefit from visual speech cues and sentence context in L2, although the benefit of visual speech cues in L2 was somewhat smaller in older adults compared to young adults.…”

“…Given that overall performance was equated in the baseline condition for both young and older bilinguals, we expect to find comparable effects of context across the age groups, as seen in Dubno et al (2000) and Buss et al (2019). Additionally, based on the results from Chauvin and Phillips (2022), we hypothesize that the benefit from visual speech cue and sentence context will be proportionally greater in L2 compared to L1 for both age groups.…”

“…Additionally, equating for overall performance at baseline in L1 allowed for a direct investigation of L2 speech perception in noise given comparable baseline L1 performance. Data and results from the young bilingual participants were previously reported in Chauvin and Phillips (2022) and are re-analyzed here to investigate age differences in the benefit of visual speech cues and/or sentence context. This document is copyrighted by the American Psychological Association or one of its allied publishers.…”

Bilingual infants readily orient to novel visual stimuli.

“…Although the aforementioned studies in speech perception have focused on auditory experience, language experience is a multimodal event that not only involves auditory experience, but also modifies the visual environment. Past studies have demonstrated that more so than monolingual learners, bilingual learners are more dependent on visual information in language input (Chauvin & Phillips, 2022; Sebastián-Gallés et al, 2012), raising the possibility that visual information may be harnessed to a greater extent in bilingual learners. As a result, we would expect effects of bilingualism on perception and learning to span the different modalities within which language presents itself.…”

Age of Acquisition Modulates Alpha Power During Bilingual Speech Comprehension in Noise

“…More recently, Krizman et al (2017) showed that bilinguals performed worse when perceiving sentences in noise in their L2 compared to monolinguals, whereas bilinguals performed better than monolinguals at perceiving tones in noise, suggesting that effects of acoustic degradation on L2 speech comprehension are dependent on linguistic knowledge. In contrast, other research has shown that bilinguals may benefit from a contextually supportive sentence context to a greater extent in their L2 compared to their L1 when listening to speech-in-noise ( Chauvin and Phillips, in press ).…”