Semantic Cues Modulate Children’s and Adults’ Processing of Audio-Visual Face Mask Speech

Abstract

During the COVID-19 pandemic, questions have been raised about the impact of face masks on communication in classroom settings. However, it is unclear to what extent visual obstruction of the speaker’s mouth or changes to the acoustic signal lead to speech processing difficulties, and whether these effects can be mitigated by semantic predictability, i.e., the availability of contextual information. The present study investigated the acoustic and visual effects of face masks on speech intelligibility and processing speed under varying semantic predictability. Twenty-six children (aged 8-12) and twenty-six adults performed an internet-based cued shadowing task, in which they had to repeat aloud the last word of sentences presented in audio-visual format. The results showed that children and adults made more mistakes and responded more slowly when listening to face mask speech compared to speech produced without a face mask. Adults were only significantly affected by face mask speech when both the acoustic and the visual signal were degraded. While acoustic mask effects were similar for children, removal of visual speech cues through the face mask affected children to a lesser degree. However, high semantic predictability reduced audio-visual mask effects, leading to full compensation of the acoustically degraded mask speech in the adult group. Even though children did not fully compensate for face mask speech with high semantic predictability, overall, they still profited from semantic cues in all conditions. Therefore, in classroom settings, strategies that increase contextual information such as building on students’ prior knowledge, using keywords, and providing visual aids, are likely to help overcome any adverse face mask effects.

Children's experiences of mask‐wearing: a systemic review and narrative synthesis

“…A similar approach could be used in establishing the effects of masks on children's ability to read others‘ facial expressions and process speech. The literature has been variable, with some studies implying a significant impact on children and others implying a negligible effect 27,28,30,34,37,39,50,53,55,56,60,64 . Interestingly, while Ruba et al 53 found that children were slightly more accurate in inferring emotions when faces were uncovered, compared with when reviewing pictures of faces with masks, there was no difference in their accuracy between faces with sunglasses and masks.…”

Effects of face masks on language comprehension in bilingual children

“…First, we examined one aspect of linguistic communication: spoken word recognition. Prior studies demonstrate that children do not show a mask detriment when they can leverage semantic knowledge, as in Schwarz et al, 2022. It is possible that other aspects of language learning, such as learning new words, are more constrained by the use of face masks.…”

“…Importantly, there is evidence that children may even be more resilient to masked communication than adults. For example, in a language comprehension task, children were better able to compensate for visual distortion caused by masks than adults (Schwarz et al., 2022). In sum, empirical evidence attesting to effects of masks on communication is not always consistent with preconceived notions about the impact of masks.…”

Language proficiency modulates listeners’ selective attention to a talker’s mouth: A conceptual replication of Birulés et al. (2020)

“…Yet this inquiry also appears of relevance to the wider field of second language acquisition (SLA) and applied linguistics, where research on L2 listening comprehension has shown benefits of visual speech cues in both laboratory and classroom contexts (e.g., Hardison, 1996Hardison, , 2003Sueyoshi & Hardison, 2005), and the role of visuals in language assessment is an area of active inquiry (for review, see Suvorov & He, 2022). Furthermore, from a more applied perspective, the wearing of face masks during the COVID-19 pandemic has drawn broader attention to the role of visual speech cues from the face during communicative interactions in general and in educational settings in particular (e.g., Schwarz et al, 2022). In the context of equity in education, understanding if and how students with lower proficiency in the language of instruction might be disproportionately influenced by teachers wearing face masks has practical implications for deciding to what extent such students may be in need of extra support.…”