Eye see what you're saying: Contrastive use of beat gesture and pitch accent affects online interpretation of spoken discourse.

Abstract

Cues to prominence such as beat gesture and contrastive pitch accent play an important role in constraining what is remembered. However, it is currently unclear how beat gesture affects online discourse processing alone and in combination with contrastive accenting. Using an adaptation of the visual world eye-tracking paradigm, we orthogonally manipulated the presence of these cues and their felicity (match) with contrast within local (sentence-level) and global (experiment-level) referential contexts. In Experiment 1, in which beat gesture and contrastive accenting were always globally felicitous with the context of filler referring expressions, beat gesture increased anticipation of both target and competitor referents of locally infelicitous critical referring expressions differing in color and shape, whereas contrastive accenting hindered resolution of these expressions. In Experiment 2, in which beat gesture and contrastive accenting were always globally infelicitous with the context of filler referring expressions, beat gesture increased anticipation of both target and competitor referents of locally felicitous critical referring expressions contrasting in color, whereas contrastive accenting did not affect their interpretation. Taken together, these findings indicate that local and global felicity of cues to prominence with contrast affects their interpretation during online spoken discourse processing.

Development of sensitivity to beat gesture and contrastive accenting in support of word learning in early childhood in boys and girls

“…With respect to contrastive accenting, previous research has shown that additional support from discourse is needed for young children to recognize discourse with contrastive accenting (Ito et al, 2014;Sekerina & Trueswell, 2012;Wells et al, 2004), and the only other cue to contrast available in discourse in the current study was the words 'but not'. Moreover, a key difference between the current study and previous research showing effects of these cues in early childhood (Austin & Sweller, 2014;Igualada et al, 2017;Lee & Snedeker, 2016;Llanes-Coromina et al, 2018;Macoun & Sweller, 2016;Vilà-Giménez et al, 2019) and adulthood (Kushch et al, 2018;Morett et al, 2021;Morett & Fraundorf, 2019;Morett, Roche, et al, 2020) is that the current study examined their effects in the context of a novel word learning task. Unlike the discourse memory tasks used in many previous studies, which require information from a coherent discourse to be remembered, the novel word learning task requires the phonological forms of novel words to be mapped onto their referents and for this mapping to be remembered.…”

“…In addition to hand flicks, beat gestures may take the form of an open palm facing outwards, a closed fist, a raised index finger, or a precision grip, all of which convey different epistemic effects (Lempert, 2011;Maricchiolo et al, 2009;Shattuck-Hufnagel et al, 2016;Streeck, 2008). Moreover, some beat gestures are not temporally aligned with pitch accenting (Rohrer et al, 2019;Shattuck-Hufnagel & Ren, 2018), and beat gestures can convey certain aspects of meaning in conjunction with concurrent speech (Alexanderson et al, 2013;Gluhareva & Prieto, 2017;Krahmer et al, 2002;Morett, Roche et al, 2020;Morett et al, 2021;Morett & Fraundorf, 2019;Yap et al, 2018). Critical to the purpose of the current work, when beat gesture consistently co-occurs with contrastive alternatives, it is interpreted as a cue to contrast in English as well as Catalan (Llanes-Coromina et al, 2018;Morett et al, 2021;Morett, Roche, et al, 2020).…”

“…Moreover, some beat gestures are not temporally aligned with pitch accenting (Rohrer et al, 2019;Shattuck-Hufnagel & Ren, 2018), and beat gestures can convey certain aspects of meaning in conjunction with concurrent speech (Alexanderson et al, 2013;Gluhareva & Prieto, 2017;Krahmer et al, 2002;Morett, Roche et al, 2020;Morett et al, 2021;Morett & Fraundorf, 2019;Yap et al, 2018). Critical to the purpose of the current work, when beat gesture consistently co-occurs with contrastive alternatives, it is interpreted as a cue to contrast in English as well as Catalan (Llanes-Coromina et al, 2018;Morett et al, 2021;Morett, Roche, et al, 2020). Thus, some contemporary researchers have proposed that all gestures, including beats, may have both prosodic and pragmatic components and can express a range of functions (Prieto et al, 2018;Shattuck-Hufnagel & Prieto, 2019).…”

“…When pitch accenting and beat gesture consistently co-occur with contrasting alternatives in discourse (e.g. THIS vs that), they serve as reliable cues to contrast, enhancing attention to and memory for them in adults (Fraundorf et al, 2010;Kushch & Prieto, 2016;Morett et al, 2021;Morett & Fraundorf, 2019;Morett, Roche, et al, 2020) as well as children (Igualada et al, 2017;Ito et al, 2012Ito et al, , 2014Lee & Snedeker, 2016;Llanes-Coromina et al, 2018;Sekerina & Trueswell, 2012). At present, however, it is uncertain whether pitch accenting and beat gesture can facilitate acquisition of contrasting sets of novel words differing in meaning, so their utility in the context of word learning is unclear.…”

Mapping Pitch Accents to Memory Representations in Spoken Discourse Among Chinese Learners of English: Effects of L2 Proficiency and Working Memory

“…When prosodic cues match the content of speech, they enhance L2 speakers’ discourse comprehension and memory ( Lee and Fraundorf, 2019 , 2021 ). For L1 speakers, when prosodic cues fail to match the content of speech, they interfere with discourse content and memory ( Harrington, 1992 ; van den Noort et al, 2006 ; Morett and Fraundorf, 2019 ; Morett et al, 2020 , 2021 ). At present, it is unclear whether this is the case for L2 speakers, however.…”