Multimodal Language Acquisition Based on Motor Learning and Interaction

Hörnstein, Jonas; Gustavsson, Lisa; Santos-Victor, José; Lacerda, Francisco

doi:10.1007/978-3-642-05181-4_20

Cited by 4 publications

(3 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their model could perform speech segmentation, lexical acquisition, and visual categorization. Hörnstein et al ( 2010 ) proposed a method based on pattern recognition and hierarchical clustering that mimics a human infant to enable a humanoid robot to acquire language. Their method allowed the robot to acquire phonemes and words from visual and auditory information through interaction with the human.…”

Section: Related Workmentioning

confidence: 99%

Cross-Situational Learning with Bayesian Generative Models for Multimodal Category and Word Learning in Robots

Taniguchi

Cangelosi

2017

Front. Neurorobot.

View full text Add to dashboard Cite

In this paper, we propose a Bayesian generative model that can form multiple categories based on each sensory-channel and can associate words with any of the four sensory-channels (action, position, object, and color). This paper focuses on cross-situational learning using the co-occurrence between words and information of sensory-channels in complex situations rather than conventional situations of cross-situational learning. We conducted a learning scenario using a simulator and a real humanoid iCub robot. In the scenario, a human tutor provided a sentence that describes an object of visual attention and an accompanying action to the robot. The scenario was set as follows: the number of words per sensory-channel was three or four, and the number of trials for learning was 20 and 40 for the simulator and 25 and 40 for the real robot. The experimental results showed that the proposed method was able to estimate the multiple categorizations and to learn the relationships between multiple sensory-channels and words accurately. In addition, we conducted an action generation task and an action description task based on word meanings learned in the cross-situational learning scenario. The experimental results showed that the robot could successfully use the word meanings learned by using the proposed method.

show abstract

Section: Related Workmentioning

confidence: 99%

Cross-Situational Learning with Bayesian Generative Models for Multimodal Category and Word Learning in Robots

Taniguchi

Cangelosi

2017

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…The number of words in a sentence at time t is denoted as B t . The generative model of the proposed method is defined as equation (1)(2)(3)(4)(5)(6)(7)(8)(9)(10).…”

Section: A Generative Modelmentioning

confidence: 99%

Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentences

Taniguchi

Inamura

2016

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Abstract-In this paper, we propose a novel unsupervised learning method for the lexical acquisition of words related to places visited by robots, from human continuous speech signals. We address the problem of learning novel words by a robot that has no prior knowledge of these words except for a primitive acoustic model. Further, we propose a method that allows a robot to effectively use the learned words and their meanings for self-localization tasks. The proposed method is nonparametric Bayesian spatial concept acquisition method (SpCoA) that integrates the generative model for self-localization and the unsupervised word segmentation in uttered sentences via latent variables related to the spatial concept. We implemented the proposed method SpCoA on SIGVerse, which is a simulation environment, and TurtleBot 2, which is a mobile robot in a real environment. Further, we conducted experiments for evaluating the performance of SpCoA. The experimental results showed that SpCoA enabled the robot to acquire the names of places from speech sentences. They also revealed that the robot could effectively utilize the acquired spatial concepts and reduce the uncertainty in self-localization.

show abstract

“…First, the chapter from [33] is about language acquisition in humanoid robots, based on interaction with a caregiver and using as few built in a priori knowledge or primitives as possible. The importance of motor learning in the language acquisition process is underlined.…”

Section: Imitation and Interaction Learningmentioning

confidence: 99%

From Motor Learning to Interaction Learning in Robots

Sigaud

Peters

2010

Studies in Computational Intelligence

View full text Add to dashboard Cite

-The number of advanced robot systems has been increasing in recent years yielding a large variety of versatile designs with many degrees of freedom. These robots have the potential of being applicable in uncertain tasks outside well-structured industrial settings. However, the complexity of both systems and tasks is often beyond the reach of classical robot programming methods. As a result, a more autonomous solution for robot task acquisition is needed where robots adaptively adjust their behaviour to the encountered situations and required tasks. Learning approaches pose one of the most appealing ways to achieve this goal. However, while learning approaches are of high importance for robotics, we cannot simply use off-the-shelf methods from the machine learning community as these usually do not scale into the domains of robotics due to excessive computational cost as well as a lack of scalability. Instead, domain appropriate approaches are needed. We focus here on several core domains of robot learning. For accurate task execution, we need motor learning capabilities. For fast learning of the motor tasks, imitation learning offers the most promising approach. Self improvement requires reinforcement learning approaches that scale into the domain of complex robots. Finally, for efficient interaction of humans with robot systems, we will need a form of interaction learning. This contribution provides a general introduction to these issues and briefly presents the contributions of the related book chapters to the corresponding research topics.

show abstract

Multimodal Language Acquisition Based on Motor Learning and Interaction

Cited by 4 publications

References 51 publications

Cross-Situational Learning with Bayesian Generative Models for Multimodal Category and Word Learning in Robots

Cross-Situational Learning with Bayesian Generative Models for Multimodal Category and Word Learning in Robots

Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentences

From Motor Learning to Interaction Learning in Robots

Contact Info

Product

Resources

About