Posture Affects How Robots and Infants Map Words to Objects

Abstract

For infants, the first problem in learning a word is to map the word to its referent; a second problem is to remember that mapping when the word and/or referent are again encountered. Recent infant studies suggest that spatial location plays a key role in how infants solve both problems. Here we provide a new theoretical model and new empirical evidence on how the body – and its momentary posture – may be central to these processes. The present study uses a name-object mapping task in which names are either encountered in the absence of their target (experiments 1–3, 6 & 7), or when their target is present but in a location previously associated with a foil (experiments 4, 5, 8 & 9). A humanoid robot model (experiments 1–5) is used to instantiate and test the hypothesis that body-centric spatial location, and thus the bodies’ momentary posture, is used to centrally bind the multimodal features of heard names and visual objects. The robot model is shown to replicate existing infant data and then to generate novel predictions, which are tested in new infant studies (experiments 6–9). Despite spatial location being task-irrelevant in this second set of experiments, infants use body-centric spatial contingency over temporal contingency to map the name to object. Both infants and the robot remember the name-object mapping even in new spatial locations. However, the robot model shows how this memory can emerge –not from separating bodily information from the word-object mapping as proposed in previous models of the role of space in word-object mapping – but through the body’s momentary disposition in space.

Children’s referent selection and word learning

“…In line with existing embodied explorations of word learning, which show that the spatial orientation of the body can affect the formation of label-object mappings (Morse et al, 2015;Samuelson et al, 2011), the current work demonstrates that the layout of the task environment entrains the spatial location of the robot's head, which in turn affects the amount of time required to encode all objects in the array. Our model therefore predicts that children's performance in similar tasks can be boosted or impaired by reducing or increasing the amount of time it takes to scan all possible referents of the novel words by increasing the spatial distance between them.…”

“…Recent work in computational developmental psychology suggests that the body's location in space plays an important role in young children's early word learning (Morse, Benitez, Belpaeme, Cangelosi, & Smith, 2015;Samuelson et al, 2011). These studies suggest that using an embodied system is important to understand the mechanisms driving word learning.…”

“…Thus, like children, iCub integrates visual, auditory, tactile and proprioceptive information to generate behaviour, for example auditory and visual information in a word learning task (although which modalities contribute to a given simulation are decided a priori by the modeller). Thus far, iCub has captured a range of developmental phenomena, for example motor development (Tikhanoff, Cangelosi, & Metta, 2011), visuomotor development (Shaw, Law, & Lee, 2014), intrinsically motivated exploration (Maestre, Cully, Gonzales, & Doncieux, 2015), affordancebased verb learning (Marocco, Cangelosi, Fischer, & Belpaeme, 2010), and spatiallygrounded noun learning (Morse et al, 2015; for a review see Cangelosi & Schlesinger, 2015).…”

The effects of learning and retrieval contexts on cross-situational word learning

“…A growing body of research on infant and robot learning has stressed the importance that learning cannot be dissociated from the world [26]. Our study further contributes to the literature by showing that semantically-themed learning trials provide learners with contextual support and facilitate learning.…”

Vocal Interactivity in-and-between Humans, Animals, and Robots

“…Indeed, vocal interaction almost always takes place in a multimodal context (Wermter et al, 2009;Liebal et al, 2013;Mavridis, 2014), and this means that vocalization may well be critically coordinated with other physical activities, such as gestures (Esposito and Esposito, 2011;Gillespie-Lynch et al, 2013;Wagner et al, 2014), gaze direction (Holler et al, 2014), and body posture (Morse et al, 2015). How are such multimodal behaviors orchestrated, especially in multi-agent situations?…”