Newborns are sensitive to the correspondence between auditory pitch and visuospatial elevation

Abstract

Amodal (redundant) and arbitrary cross-sensory feature associations involve the context-insensitive mapping of absolute feature values across sensory domains. Cross-sensory associations of a different kind, known as correspondences, involve the context-sensitive mapping of relative feature values. Are such correspondences in place at birth (like amodal associations), or are they learned from subsequently experiencing relevant feature co-occurrences in the world (like arbitrary associations)? To decide between these two possibilities, human newborns (median age = 44 hr) watched animations in which two balls alternately rose and fell together in space. The pitch of an accompanying sound rose and fell either congruently with this visual change (pitch rising and falling as the balls moved up and down), or incongruently (pitch rising and falling as the balls moved down and up). Newborns' looking behavior was sensitive to this congruence, providing the strongest indication to date that cross-sensory correspondences can be in place at birth.

Space-pitch associations differ in their susceptibility to language

“…People also link spatial size and pitch, associating higher pitches with smaller objects and lower pitches with larger objects (e.g., Parise & Spence, 2009). Even infants and newborns seem to be sensitive to associations between some spatial dimensions and pitch (e.g., Dolscheid, Hunnius, Casasanto, & Majid, 2014;Pietraszewski, Wertz, Bryant, & Wynn, 2017;Walker et al, 2010Walker et al, , 2018; but see Lewkowicz & Minar et al, 2014).…”

“…If so, it is also conceivable that the mappings are not equally salient to both groups-a possibility that has not been explored previously. Height-pitch associations reveal themselves in a variety of tasks (Chiou & Rich, 2012;Maeda, Kanai, & Shimojo, 2014;Rusconi et al, 2006), are present from as early as 44 hours from birth (Walker et al, 2018), and have been said to be universally present in languages (Evans & Treisman, 2010;Pratt, 1930;Stumpf, 1883). It is claimed this mapping reflects real-world auditory scene statistics-i.e., higher frequency sounds come from higher in space-and that the convoluted anatomy of the outer ear evolved to mirror these auditory statistics (Parise, Knorre, & Ernst, 2014).…”

Auditory information for spatial location and pitch–height correspondence support young infants’ perception of object persistence

“…Two-to 3-month-olds appear to associate certain colours and shapes (Wagner & Dobkins, 2011) and 4-month-olds spontaneously associate pitch (sound) with height in the visual field, and with object angularity (Walker et al, 2010) and object thickness (Dolscheid, Hunnius, Casasanto, & Majid, 2014). The pitch-height correspondence may well be fundamental because it has been detected in newborns (Walker et al, 2018). Thus, in addition to the possibility that spatially colocated auditory information may enhance 4-month-olds' perception of vertical trajectory continuity, it is also possible that presenting a sound that varies in pitch congruently with the changing vertical location of the object could have a similar effect.…”

“…This correspondence would conventionally be identified as an arbitrary relation (Bahrick, Netto, & Hernandez-Reif, 1998) or as involving modality-specific information (Bahrick, 2010), and so assumed to be necessarily a learned relation. Although there are those who argue that intersensory correspondences are learned from correlations that exist in the environment (Spence & Deroy, 2012), there is growing evidence that they exist early and without a clear basis in experience for their learning (Dolscheid et al, 2014;Walker et al, 2010;Walker et al, 2018). If the pitch-height correspondence is learned from experience, this must happen within the first few days after birth or in utero (Walker et al, 2018).…”

“…Although there are those who argue that intersensory correspondences are learned from correlations that exist in the environment (Spence & Deroy, 2012), there is growing evidence that they exist early and without a clear basis in experience for their learning (Dolscheid et al, 2014;Walker et al, 2010;Walker et al, 2018). If the pitch-height correspondence is learned from experience, this must happen within the first few days after birth or in utero (Walker et al, 2018). It appears that the appropriate pitch-height correlations may be available in the environment, because Parise, Knorre, and Ernst (2014) demonstrated that ambient sounds emanating from higher in space were on average higher in pitch than those emanating from lower in space.…”