Survey on frontiers of language and robotics

Tangiuchi, T.; Mochihashi, Daichi; Nagai, Takayuki; Uchida, Satoru; Inoue, Nozomu; Kobayashi, Ichiro; Nagaoka, Tomoaki; Hagiwara, Yoshinobu; Iwahashi, Naoto; Inamura, Tetsunari

doi:10.1080/01691864.2019.1632223

Cited by 41 publications

(27 citation statements)

References 155 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further application of Neuro-SERKET and the development of cognitive systems that enable a robot to form concepts, learn behaviors, and acquire language in a real-world environment is our future challenge. In particular, it has become clear that language learning in a realworld environment requires a wide range of cognitive capabilities [40]. For this reason, at least two additional approaches should be applied to Neuro-SERKET.…”

Section: Resultsmentioning

confidence: 99%

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

et al. 2020

View full text Add to dashboard Cite

This paper describes a framework for the development of an integrative cognitive system based on probabilistic generative models (PGMs) called Neuro-SERKET. Neuro-SERKET is an extension of SERKET, which can compose elemental PGMs developed in a distributed manner and provide a scheme that allows the composed PGMs to learn throughout the system in an unsupervised way. In addition to the head-to-tail connection supported by SERKET, Neuro-SERKET supports tail-totail and head-to-head connections, as well as neural network-based modules, i.e., deep generative models. As an example of a Neuro-SERKET application, an integrative model was developed by composing a variational autoencoder (VAE), a Gaussian mixture model (GMM), latent Dirichlet allocation (LDA), and automatic speech recognition (ASR). The model is called VAE + GMM + LDA + ASR. The performance of VAE + GMM + LDA + ASR and the validity of Neuro-SERKET were demonstrated through a multimodal categorization task using image data and a speech signal of numerical digits.

show abstract

Section: Resultsmentioning

confidence: 99%

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Hagiwara et al ( 2016 , 2018 ) proposed a Bayesian model to acquire the hierarchical structure of spatial concepts based on the sensory-motor information of a robot in real home environments. Tangiuchi et al ( 2019 ) summarized their studies and related works on cognitive developmental robotics that can learn a language from interaction with their environment and unsupervised learning methods that enable robots to learn a language without hand-crafted training data. As studies on developmental robotics (Cangelosi and Schlesinger, 2014 ), Cangelosi and his group have proposed computational models for an iCub humanoid robot to ground action words through embodied communications (Marocco et al, 2010 ; Stramandinoli et al, 2017 ; Taniguchi et al, 2017 ; Zhong et al, 2019 ).…”

Section: Related Workmentioning

confidence: 99%

Symbol Emergence as an Interpersonal Multimodal Categorization

et al. 2019

View full text Add to dashboard Cite

This study focuses on category formation for individual agents and the dynamics of symbol emergence in a multi-agent system through semiotic communication. Semiotic communication is defined, in this study, as the generation and interpretation of signs associated with the categories formed through the agent's own sensory experience or by exchange of signs with other agents. From the viewpoint of language evolution and symbol emergence, organization of a symbol system in a multi-agent system (i.e., agent society) is considered as a bottom-up and dynamic process, where individual agents share the meaning of signs and categorize sensory experience. A constructive computational model can explain the mutual dependency of the two processes and has mathematical support that guarantees a symbol system's emergence and sharing within the multi-agent system. In this paper, we describe a new computational model that represents symbol emergence in a two-agent system based on a probabilistic generative model for multimodal categorization. It models semiotic communication via a probabilistic rejection based on the receiver's own belief. We have found that the dynamics by which cognitively independent agents create a symbol system through their semiotic communication can be regarded as the inference process of a hidden variable in an interpersonal multimodal categorizer, i.e., the complete system can be regarded as a single agent performing multimodal categorization using the sensors of all agents, if we define the rejection probability based on the Metropolis-Hastings algorithm. The validity of the proposed model and algorithm for symbol emergence, i.e., forming and sharing signs and categories, is also verified in an experiment with two agents observing daily objects in the real-world environment. In the experiment, we compared three communication algorithms: no communication, no rejection, and the proposed algorithm. The experimental results demonstrate that our model reproduces the phenomena of symbol emergence, which does not require a teacher who would know a pre-existing symbol system. Instead, the multi-agent system can form and use a symbol system without having pre-existing categories.

show abstract

“…Another approach could implement in a robot the ability to explain its own actions. Little research has investigated the possibility of learning the relationship between a robot’s actions and their corresponding explanations ( Taniguchi et al, 2019 ). Platter et al proposed bidirectional mapping between the whole-body motion of a humanoid robot and language using deep recurrent networks ( Plappert et al, 2018 ).…”

Section: Technical Challenges Toward Artificial Systems That Incorporate Humanitude Techniquesmentioning

confidence: 99%

Technical Challenges for Smooth Interaction With Seniors With Dementia: Lessons From Humanitude™

et al. 2021

View full text Add to dashboard Cite

Due to cognitive and socio-emotional decline and mental diseases, senior citizens, especially people with dementia (PwD), struggle to interact smoothly with their caregivers. Therefore, various care techniques have been proposed to develop good relationships with seniors. Among them, Humanitude is one promising technique that provides caregivers with useful interaction skills to improve their relationships with PwD, from four perspectives: face-to-face interaction, verbal communication, touch interaction, and helping care receivers stand up (physical interaction). Regardless of advances in elderly care techniques, since current social robots interact with seniors in the same manner as they do with younger adults, they lack several important functions. For example, Humanitude emphasizes the importance of interaction at a relatively intimate distance to facilitate communication with seniors. Unfortunately, few studies have developed an interaction model for clinical care communication. In this paper, we discuss the current challenges to develop a social robot that can smoothly interact with PwDs and overview the interaction skills used in Humanitude as well as the existing technologies.

show abstract

Survey on frontiers of language and robotics

Cited by 41 publications

References 155 publications

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

Symbol Emergence as an Interpersonal Multimodal Categorization

Technical Challenges for Smooth Interaction With Seniors With Dementia: Lessons From Humanitude™

Contact Info

Product

Resources

About