Infant-directed visual prosody

Abstract

Acoustical changes in the prosody of mothers’ speech to infants are distinct and near universal. However, less is known about the visible properties mothers’ infant-directed (ID) speech, and their relation to speech acoustics. Mothers’ head movements were tracked as they interacted with their infants using ID speech, and compared to movements accompanying their adult-directed (AD) speech. Movement measures along three dimensions of head translation, and three axes of head rotation were calculated. Overall, more head movement was found for ID than AD speech, suggesting that mothers exaggerate their visual prosody in a manner analogous to the acoustical exaggerations in their speech. Regression analyses examined the relation between changing head position and changing acoustical pitch (F0) over time. Head movements and voice pitch were more strongly related in ID speech than in AD speech. When these relations were examined across time windows of different durations, stronger relations were observed for shorter time windows (< 5 sec). However, the particular form of these more local relations did not extend or generalize to longer time windows. This suggests that the multimodal correspondences in speech prosody are variable in form, and occur within limited time spans.

Cross-cultural evidence for multimodal motherese: Asian Indian mothers’ adaptive use of synchronous words and gestures

“…We provide a bi-directional mechanistic explanation that involves the organism and its sociolinguistic environment - the ongoing interaction between infants’ perception and maternal scaffolding (Gogate & Hollich, 2010, 2013; Sullivan & Horrowitz, 1983; Yu, Ballard & Aslin, 2005). In general, caregivers coordinate their use of higher pitch, exaggerated intonation contours, elongated speech and longer pauses between utterances (Cooper, Abraham, Berman, & Statska, 1997; Fernald & Simon, 1984; Kitamura & Burnham, 2003) with simultaneous visual mouth movements (Bahrick & Pickens, 1988; Dodd, 1979; Legerstee, 1990; Meltzoff & Kuhl, 1994), more animated head movements and facial expressions (Smith & Strader, 2014; Walker-Andrews, 1997), and gestures using hands and body (Brand, Baldwin & Ashburn, 2002; Brand & Tapscott, 2007). This coordinated information is amodal , invariant , and redundant; the same information conveyed to one sense modality is conveyed to another in the form of a common temporal structure, tempo, rhythm, and spatial colocation (see review by Gogate & Hollich, 2010).…”

The Multisensory Attention Assessment Protocol (MAAP): Characterizing individual differences in multisensory attention skills in infants and children and relations with language and cognition.

“…Only the positive social events (women speaking in affective positive IDS) were used from Experiment 1. They were judged most appropriate given IDS is highly salient to infants (Cooper & Aslin, 1990; Fernald, 1984; for a review, see Soderstrom, 2007) and it provides exaggerated intersensory information (Bahrick & Todd, 2012; Kubicek et al, 2014; Smith & Strader, 2014). Also, given our focus on assessing developmental change, positive social events were chosen as performance on these trials showed the most improvement with age in Experiment 1 (see Table 2).…”

Maternal Vocal Interactions with Infants: Reciprocal Visual Influences

“…Surprisingly, 11‐month‐olds did not respond preferentially to visual‐only versions of ID singing (Costa‐Giomi, 2014), which does not preclude the possibility that infants would do so for natural versions rather than portrayals in the absence of an infant. Mothers produce distinctive facial expressions (Chong, Werker, Russell, & Carroll, ) and head movements (Smith & Strader, ) when they speak to infants, and Experiment 1 revealed that they smile more when singing than when talking to infants. It is possible, then, that infants’ greater interest in audiovisual renditions of maternal singing (Nakata & Trehub, ) stems, in part, from more positive facial displays during such singing.…”