Symbol Emergence in Cognitive Developmental Systems: A Survey

Taniguchi, Tadahiro; Piater, Justus; Wörgötter, Florentin; Uğur, Emre; Hoffmann, Matej; Jamone, Lorenzo; Nagai, Takayuki; Rosman, Benjamin; Matsuka, Toshihiko; Iwahashi, Naoto; Öztop, Erhan

doi:10.1109/tcds.2018.2867772

Cited by 75 publications

(70 citation statements)

References 138 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where ≈∝ represents the terms "approximation" and "proportion to". 1 The decomposition is described as follows.…”

Section: Head-to-head Decompositionmentioning

confidence: 99%

“…The development of integrative cognitive systems that can form perceptual and behavioral concepts using multimodal sensorimotor information and learn and understand a language in a real-world environment is a significant challenge in artificial intelligence (AI) and robotics [1]. This paper describes a theoretical framework called Neuro-SERKET for this purpose.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

et al. 2020

Self Cite

View full text Add to dashboard Cite

This paper describes a framework for the development of an integrative cognitive system based on probabilistic generative models (PGMs) called Neuro-SERKET. Neuro-SERKET is an extension of SERKET, which can compose elemental PGMs developed in a distributed manner and provide a scheme that allows the composed PGMs to learn throughout the system in an unsupervised way. In addition to the head-to-tail connection supported by SERKET, Neuro-SERKET supports tail-totail and head-to-head connections, as well as neural network-based modules, i.e., deep generative models. As an example of a Neuro-SERKET application, an integrative model was developed by composing a variational autoencoder (VAE), a Gaussian mixture model (GMM), latent Dirichlet allocation (LDA), and automatic speech recognition (ASR). The model is called VAE + GMM + LDA + ASR. The performance of VAE + GMM + LDA + ASR and the validity of Neuro-SERKET were demonstrated through a multimodal categorization task using image data and a speech signal of numerical digits.

show abstract

“…where ≈∝ represents the terms "approximation" and "proportion to". 1 The decomposition is described as follows.…”

Section: Head-to-head Decompositionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…A concept of symbol emergence in cognitive systems is surveyed in (Tadahiro Taniguchi, 2018). A symbol emergence system is socially self-organized through both semiotic communications with autonomous cognitive developmental agents and physical interactions with the environment, as shown in Figure 1.…”

Section: Introductionmentioning

confidence: 99%

Symbol Emergence as an Interpersonal Multimodal Categorization

et al. 2019

Self Cite

View full text Add to dashboard Cite

This study focuses on category formation for individual agents and the dynamics of symbol emergence in a multi-agent system through semiotic communication. Semiotic communication is defined, in this study, as the generation and interpretation of signs associated with the categories formed through the agent's own sensory experience or by exchange of signs with other agents. From the viewpoint of language evolution and symbol emergence, organization of a symbol system in a multi-agent system (i.e., agent society) is considered as a bottom-up and dynamic process, where individual agents share the meaning of signs and categorize sensory experience. A constructive computational model can explain the mutual dependency of the two processes and has mathematical support that guarantees a symbol system's emergence and sharing within the multi-agent system. In this paper, we describe a new computational model that represents symbol emergence in a two-agent system based on a probabilistic generative model for multimodal categorization. It models semiotic communication via a probabilistic rejection based on the receiver's own belief. We have found that the dynamics by which cognitively independent agents create a symbol system through their semiotic communication can be regarded as the inference process of a hidden variable in an interpersonal multimodal categorizer, i.e., the complete system can be regarded as a single agent performing multimodal categorization using the sensors of all agents, if we define the rejection probability based on the Metropolis-Hastings algorithm. The validity of the proposed model and algorithm for symbol emergence, i.e., forming and sharing signs and categories, is also verified in an experiment with two agents observing daily objects in the real-world environment. In the experiment, we compared three communication algorithms: no communication, no rejection, and the proposed algorithm. The experimental results demonstrate that our model reproduces the phenomena of symbol emergence, which does not require a teacher who would know a pre-existing symbol system. Instead, the multi-agent system can form and use a symbol system without having pre-existing categories.

show abstract

“…Then, we define a position distribution as the spatial extent representing a place in the environment. Our study regarding the spatial concept formation and the lexical acquisition also constitute constructive approaches to the human developmental process and symbol emergence in cognitive developmental systems (Cangelosi and Schlesinger 2015;Taniguchi et al 2018b). Thus, we assume that the robot has not acquired any vocabulary in advance and can recognize only phonemes or syllables.…”

Section: Introductionmentioning

confidence: 95%

Improved and scalable online learning of spatial concepts and language models with mapping

et al. 2020

Self Cite

View full text Add to dashboard Cite

We propose a novel online learning algorithm, called SpCoSLAM 2.0, for spatial concepts and lexical acquisition with high accuracy and scalability. Previously, we proposed SpCoSLAM as an online learning algorithm based on unsupervised Bayesian probabilistic model that integrates multimodal place categorization, lexical acquisition, and SLAM. However, our original algorithm had limited estimation accuracy owing to the influence of the early stages of learning, and increased computational complexity with added training data. Therefore, we introduce techniques such as fixed-lag rejuvenation to reduce the calculation time while maintaining an accuracy higher than that of the original algorithm. The results show that, in terms of estimation accuracy, the proposed algorithm exceeds the original algorithm and is comparable to batch learning. In addition, the calculation time of the proposed algorithm does not depend on the amount of training data and becomes constant for each step of the scalable algorithm. Our approach will contribute to the realization of long-term spatial language interactions between humans and robots.

show abstract

Symbol Emergence in Cognitive Developmental Systems: A Survey

Cited by 75 publications

References 138 publications

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

Symbol Emergence as an Interpersonal Multimodal Categorization

Improved and scalable online learning of spatial concepts and language models with mapping

Contact Info

Product

Resources

About