Multimodal word learning from Infant Directed Speech

Abstract

Abstract-When adults talk to infants they do that in a different way compared to how they communicate with other adults. This kind of Infant Directed Speech (IDS) typically highlights target words using focal stress and utterance final position. Also, speech directed to infants often refers to objects, people and events in the world surrounding the infant. Because of this, the sound sequences the infant hears are very likely to co-occur with actual objects or events in the infant's visual field. In this work we present a model that is able to learn wordlike structures from multimodal information sources without any pre-programmed linguistic knowlege, by taking advantage of the characteristics of IDS. The model is implemented on a humanoid robot platform and is able to extract wordlike patterns and associating these to objects in the visual surrounding.

Goal emulation and planning in perceptual space using learned affordances

“…The robot platform consists of a five fingered 16 DOF robot hand 2 and a 7 DOF robot arm. 3 For robot perception, an infrared range camera, 4 with 176 × 144 pixel array, 0.23°an-gular resolution and 1 cm distance accuracy was used. Along with the range image, the camera also provides greyscale image of the scene and a confidence value for each pixel.…”

“…In the last decade, with similar views in mind, various developmental stages have been studied, modeled and transferred to robots. These stages correspond to acquisition of skills at different levels and ages, ranging from the emergence of motor patterns before birth [1] and the development of pattern generators for crawling [2] to language learning [3] (see [4] for a comprehensive review).…”

Learning words and speech units through natural interactions

“…By doing this we get a measurement of the visual similarity between objects that is invariant to both scale and rotation. For details please refer to [8].…”