Robust comprehension of natural language instructions by a domestic service robot

Kobori, Takahiro; Nagaoka, Tomoaki; Nakano, Mikio; Nagai, Takayuki; Iwahashi, Naoto; Funakoshi, Kotaro; Kaneko, Masahide

doi:10.1080/01691864.2016.1252689

Cited by 7 publications

(1 citation statement)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interactive robots that cooperate with humans must take appropriate actions in response to user requests. Existing human-computer interaction systems, which cooperate with human users and include interactive robots, assume that users explicitly verbalize their requests and that systems can confirm ambiguous requests with the users [1], [2], [3]. However, user requests often have information gaps in their actual demands because it is challenging to verbalize requirements without over-or underestimating them [4], [5].…”

Section: Introductionmentioning

confidence: 99%

Do as I Demand, Not as I Say: A Dataset for Developing a Reflective Life-Support Robot

Tanaka,

Yamasaki,

Yuguchi

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Interactive robots that cooperate with humans must take appropriate actions in response to their requests. Unfortunately, such requests often have information gaps with their actual demands. However, robots are still expected to reason and act on what is required, depending on the situation. We call these reflective actions. To achieve such reflective actions for robots, we constructed a dataset that consists of the reflective actions of a domestic manipulation robot, in which the actions correspond to user utterances with their surroundings situations. By crowdsourcing, we defined several action scenarios that could be regarded as reflective. We recorded videos of situations described in the crowdsourcing scenarios, corresponding to the user situations just before the robot's reflective actions. We also annotated the videos of the user utterance transcriptions, objects, user poses, and user positions to investigate the contribution of such descriptive features to the reflective action decisions. Our experimental results indicated that even though our newly defined task is very challenging, it can be solved if the system has a concrete understanding of the situation.

show abstract

Section: Introductionmentioning

confidence: 99%

Do as I Demand, Not as I Say: A Dataset for Developing a Reflective Life-Support Robot

Tanaka,

Yamasaki,

Yuguchi

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

L2R-Nav: A Large Language Model-Enhanced Framework for Robotic Navigation

Wu,

Li,

Chen

et al. 2024

Knowledge Science, Engineering and Management

View full text Add to dashboard Cite

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

et al. 2020

Self Cite

View full text Add to dashboard Cite

This paper describes a framework for the development of an integrative cognitive system based on probabilistic generative models (PGMs) called Neuro-SERKET. Neuro-SERKET is an extension of SERKET, which can compose elemental PGMs developed in a distributed manner and provide a scheme that allows the composed PGMs to learn throughout the system in an unsupervised way. In addition to the head-to-tail connection supported by SERKET, Neuro-SERKET supports tail-totail and head-to-head connections, as well as neural network-based modules, i.e., deep generative models. As an example of a Neuro-SERKET application, an integrative model was developed by composing a variational autoencoder (VAE), a Gaussian mixture model (GMM), latent Dirichlet allocation (LDA), and automatic speech recognition (ASR). The model is called VAE + GMM + LDA + ASR. The performance of VAE + GMM + LDA + ASR and the validity of Neuro-SERKET were demonstrated through a multimodal categorization task using image data and a speech signal of numerical digits.

show abstract

Robust comprehension of natural language instructions by a domestic service robot

Cited by 7 publications

References 7 publications

Do as I Demand, Not as I Say: A Dataset for Developing a Reflective Life-Support Robot

Do as I Demand, Not as I Say: A Dataset for Developing a Reflective Life-Support Robot

L2R-Nav: A Large Language Model-Enhanced Framework for Robotic Navigation

Neuro-SERKET: Development of Integrative Cognitive System Through the Composition of Deep Probabilistic Generative Models

Contact Info

Product

Resources

About