Mutual learning of an object concept and language model based on MLDA and NPYLM

Nagaoka, Tomoaki; Nagai, Takayuki; Funakoshi, Kotaro; Nagasaka, Shogo; Taniguchi, Tadahiro; Iwahashi, Naoto

doi:10.1109/iros.2014.6942621

Cited by 35 publications

(28 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• The extension for a mutual segmentation model of sound strings and situations based on multimodal information will be achieved based on a multimodal LDA with nested Pitman-Yor language model (Nakamura et al, 2014) and a spatial concept acquisition model that integrates self-localization and unsupervised word discovery from spoken sentences (Taniguchi et al, 2016a).…”

Section: Discussionmentioning

confidence: 99%

“…As shown in Figure 2, the model generating observations through categories on each sensor from an integrated concept in an agent can be extended as the model generating observations through categories on each agent from a word in a multi-agent system. Figure 2 (a) represents a graphical model for probabilistic generative model multimodal categorization, e.g., Nakamura et al (2014). It can integrate multimodal information, e.g., haptics and visual information, and form categories.…”

Section: Expansion Of a Multimodal Categorizer From Personal To Intermentioning

confidence: 99%

“…Constructive approaches are highly advantageous to understanding complex systems and are also useful for studying evolutionary linguistics (Steels, 1997;Hashimoto, 1999). Nakamura et al (2014); Taniguchi et al (2018) developed integrative probabilistic generative models for multimodal categorization and word discovery by a robot (using multimodal sensorimotor information obtained by a robot, and speech signals). However, their model only explains individual learning, and they implicitly presume that a teacher has a fixed and static symbol system.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Symbol Emergence as an Interpersonal Multimodal Categorization

et al. 2019

Self Cite

View full text Add to dashboard Cite

This study focuses on category formation for individual agents and the dynamics of symbol emergence in a multi-agent system through semiotic communication. Semiotic communication is defined, in this study, as the generation and interpretation of signs associated with the categories formed through the agent's own sensory experience or by exchange of signs with other agents. From the viewpoint of language evolution and symbol emergence, organization of a symbol system in a multi-agent system (i.e., agent society) is considered as a bottom-up and dynamic process, where individual agents share the meaning of signs and categorize sensory experience. A constructive computational model can explain the mutual dependency of the two processes and has mathematical support that guarantees a symbol system's emergence and sharing within the multi-agent system. In this paper, we describe a new computational model that represents symbol emergence in a two-agent system based on a probabilistic generative model for multimodal categorization. It models semiotic communication via a probabilistic rejection based on the receiver's own belief. We have found that the dynamics by which cognitively independent agents create a symbol system through their semiotic communication can be regarded as the inference process of a hidden variable in an interpersonal multimodal categorizer, i.e., the complete system can be regarded as a single agent performing multimodal categorization using the sensors of all agents, if we define the rejection probability based on the Metropolis-Hastings algorithm. The validity of the proposed model and algorithm for symbol emergence, i.e., forming and sharing signs and categories, is also verified in an experiment with two agents observing daily objects in the real-world environment. In the experiment, we compared three communication algorithms: no communication, no rejection, and the proposed algorithm. The experimental results demonstrate that our model reproduces the phenomena of symbol emergence, which does not require a teacher who would know a pre-existing symbol system. Instead, the multi-agent system can form and use a symbol system without having pre-existing categories.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Expansion Of a Multimodal Categorizer From Personal To Intermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Symbol Emergence as an Interpersonal Multimodal Categorization

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…As a method that further improves the performance of the lexical acquisition, a mutual learning method was proposed by Nakamura et al on the basis of the integration of the learning of object concepts with a language model [35], [36]. Following a similar approach, Heymann et al proposed a method that alternately and repeatedly updates phoneme recognition results and the language model by using unsupervised word segmentation [37].…”

Section: Discussionmentioning

confidence: 99%

Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentences

Taniguchi

Inamura

2016

IEEE Trans. Cogn. Dev. Syst.

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper, we propose a novel unsupervised learning method for the lexical acquisition of words related to places visited by robots, from human continuous speech signals. We address the problem of learning novel words by a robot that has no prior knowledge of these words except for a primitive acoustic model. Further, we propose a method that allows a robot to effectively use the learned words and their meanings for self-localization tasks. The proposed method is nonparametric Bayesian spatial concept acquisition method (SpCoA) that integrates the generative model for self-localization and the unsupervised word segmentation in uttered sentences via latent variables related to the spatial concept. We implemented the proposed method SpCoA on SIGVerse, which is a simulation environment, and TurtleBot 2, which is a mobile robot in a real environment. Further, we conducted experiments for evaluating the performance of SpCoA. The experimental results showed that SpCoA enabled the robot to acquire the names of places from speech sentences. They also revealed that the robot could effectively utilize the acquired spatial concepts and reduce the uncertainty in self-localization.

show abstract

“…One interesting extension of this model concerns the selfacquisition of language combining automatic speech recognition with the MLDA system. Here, unsupervised morphological analysis [85] is performed on phoneme recognition results in order to acquire a vocabulary. The point of this model is that multimodal categories are used for learning lexical information and vice versa.…”

Section: Unsupervised Learning Viewpoint: From Multimodal Categorimentioning

confidence: 99%

Symbol Emergence in Cognitive Developmental Systems: A Survey

Taniguchi

Piater

Wörgötter

et al. 2019

IEEE Trans. Cogn. Dev. Syst.

Self Cite

View full text Add to dashboard Cite

Humans use signs, e.g., sentences in a spoken language, for communication and thought. Hence, symbol systems like language are crucial for our communication with other agents and adaptation to our real-world environment. The symbol systems we use in our human society adaptively and dynamically change over time. In the context of artificial intelligence (AI) and cognitive systems, the symbol grounding problem has been regarded as one of the central problems related to symbols. However, the symbol grounding problem was originally posed to connect symbolic AI and sensorimotor information and did not consider many interdisciplinary phenomena in human communication and dynamic symbol systems in our society, which semiotics considered. In this paper, we focus on the symbol emergence problem, addressing not only cognitive dynamics but also the dynamics of symbol systems in society, rather than the symbol grounding problem. We first introduce the notion of a symbol in semiotics from the humanities, to leave the very narrow idea of symbols in symbolic AI. Furthermore, over the years, it became more and more clear that symbol emergence has to be regarded as a multifaceted problem. Therefore, secondly, we review the history of the symbol emergence problem in different fields, including both biological and artificial systems, showing their mutual relations. We summarize the discussion and provide an integrative viewpoint and comprehensive overview of symbol emergence in cognitive systems. Additionally, we describe the challenges facing the creation of cognitive systems that can be part of symbol emergence systems. Fig. 1. Robot in a home environment that has to deal with complex manipulation, planning, and interaction via semiotic communication with human users.

show abstract

Mutual learning of an object concept and language model based on MLDA and NPYLM

Cited by 35 publications

References 10 publications

Symbol Emergence as an Interpersonal Multimodal Categorization

Symbol Emergence as an Interpersonal Multimodal Categorization

Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentences

Symbol Emergence in Cognitive Developmental Systems: A Survey

Contact Info

Product

Resources

About