Leveraging audiovisual speech perception to measure anticipatory coarticulation

Abstract

A noninvasive method for accurately measuring anticipatory coarticulation at experimentally defined temporal locations is introduced. The method leverages work in audiovisual (AV) speech perception to provide a synthetic and robust measure that can be used to inform psycholinguistic theory. In this validation study, speakers were audio-video recorded while producing simple subject-verb-object sentences with contrasting object noun rhymes. Coarticulatory resistance of target noun onsets was manipulated as was metrical context for the determiner that modified the noun. Individual sentences were then gated from the verb to sentence end at segmental landmarks. These stimuli were presented to perceivers who were tasked with guessing the sentence-final rhyme. An audio-only condition was included to estimate the contribution of visual information to perceivers' performance. Findings were that perceivers accurately identified rhymes earlier in the AV condition than in the audio-only condition (i.e., at determiner onset vs determiner vowel). Effects of coarticulatory resistance and metrical context were similar across conditions and consistent with previous work on coarticulation. These findings were further validated with acoustic measurement of the determiner vowel and a cumulative video-based measure of perioral movement. Overall, gated AV speech perception can be used to test specific hypotheses regarding coarticulatory scope and strength in running speech. V

A psycholinguistic method for measuring coarticulation in child and adult speech

“…The accuracy results from the forward-gated AV speech prediction task (Experiment 1) replicate findings from our prior study (Redford et al, 2018) and easily conform with expectations based on more than a half-century of study on coarticulation, including more recent work: vowel-to-vowel coarticulation spans syllable and word boundaries (e.g., Magen, 1997;Goffman et al, 2008;Grosvald, 2009;; it may however be "blocked" by stressed syllables (see, e.g., Fowler, 1981). The accuracy results also suggest minimal to no age-related difference in coarticulatory scope given a simple elicitation task.…”

“…Just as a lack of experience with children's speech may adversely impact the ability of college-aged adults to reliably predict sounds from the coarticulatory cues provided in AV speech, so too might perceiver performance be affected by the naturalness of the prediction task. The forward prediction task that we validated in Redford et al (2018) is a reasonably natural task in that there is ample psycholinguistic evidence to suggest that listeners make use of prediction to facilitate speech processing and language comprehension (see, e.g., Federmeier, 2007;Pickering & Garrod, 2007), including the use of anticipatory cues to speech sound articulation (Salverda, Kleinschmidt, & Tanenhaus, 2014). Forward prediction also aligns perfectly with the goal of measuring anticipatory coarticulation.…”

“…In previous work, we demonstrated that aggregated perceiver responses in a forward-gated audiovisual (AV) speech prediction task yields a measure of coarticulatory onset and strength in adult speech that conforms with literature-based expectations and direct video-based measurement of orofacial movement (Redford, Kallay, Bogdanov, & Vatikiotis-Bateson, 2018). As in other production studies, the gated AV speech method for measuring coarticulation requires that speakers are recorded while producing several repetitions of sentences that contain contrasts meant to elicit coarticulatory effects of interest.…”

“…Also, direct measurement methods may adversely impact the naturalness of spoken language, limiting the usefulness of these methods for the psychological study of coarticulation, especially in populations with immature or low inhibitory control. Accordingly, we have offered an alternative, psycholinguistic method for measuring speech movements and sound shaping associated with coarticulation (Redford et al, 2018). The results from the current study indicate that the method is highly reliable when used to measure anticipatory coarticulation regardless of a speaker's age.…”

“…One group of perceivers completed the task in the laboratory, another completed the task online. Predictions based on prior work were that accuracy would be higher at gates closer to the target noun vowel than further away, and already above chance at the determiner onset (Redford et al, 2018). We also expected an effect of age group on prediction accuracy based either on age-related differences in the temporal extent and strength of anticipatory coarticulation and/or college-aged adult perceivers' lack of experience with children's speech.…”