Back to the Blocks World: Learning New Actions through Situated Human-Robot Dialogue

She, Lanbo; Yang, Shaolong; Cheng, Yu; Jia, Yunyi; Chai, Joyce; Xi, Ning

doi:10.3115/v1/w14-4313

Cited by 58 publications

(38 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to previous works (She et al, 2014b;Misra et al, 2015), our approach has three unique characteristics. First, rather than a single goal state associated with a verb, our approach captures a space of hypotheses which can potentially account for a wider range of novel situations when the verb is applied.…”

Section: Introductionmentioning

confidence: 88%

“…To support action learning, previously we have developed a system where the robot can acquire the meaning of a new verb (e.g., stack) by following human's step-by-step language instructions (She et al, 2014a;She et al, 2014b). By performing the actions at each step, the robot is able to acquire the desired goal state associated with the new verb.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction

She

Chai²

2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Self Cite

View full text Add to dashboard Cite

As a new generation of cognitive robots start to enter our lives, it is important to enable robots to follow human commands and to learn new actions from human language instructions. To address this issue, this paper presents an approach that explicitly represents verb semantics through hypothesis spaces of fluents and automatically acquires these hypothesis spaces by interacting with humans. The learned hypothesis spaces can be used to automatically plan for lower-level primitive actions towards physical world interaction. Our empirical results have shown that the representation of a hypothesis space of fluents, combined with the learned hypothesis selection algorithm, outperforms a previous baseline. In addition, our approach applies incremental learning, which can contribute to life-long learning from humans in the future.

show abstract

Section: Introductionmentioning

confidence: 88%

Section: Related Workmentioning

confidence: 99%

Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction

She

Chai²

2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Some pair language and robot actions, then build models for mapping instructions to actions [1,2]. In another study, robots are enabled to learn actions and the lexicons referring to those actions [3]. Researchers have also made use of semantic parsing.…”

Section: Related Workmentioning

confidence: 99%

A Natural Language Control System for Application Specific Robots

Jajoo¹,

Supekar²,

Hegde³

2017

IJCA

View full text Add to dashboard Cite

Due to the increasing presence of robots in several industries, human-robot interaction using natural language has become an important research area. The concept of controlling robots by transforming language instructions in English into executable code for robots is discussed. The proposed approach makes use of semantic similarity by comparing the given instruction with those within a corpus and executes the instruction most similar to the one given. The method is application specific but the modular nature of the system allows it to be adapted for any robot and for any purpose. For this project, a Raspberry Pi based robot following navigation commands is used for experiments and a success rate of 96% is observed.

show abstract

“…Providing machines with the ability to understand natural language commands is a key component for a natural human-robot interaction. For example, "Back to the blocks world" (She et al, 2014) and "Tell me Dave" (Misra et al, 2015) focused on learning the natural language commands for simple manipulation tasks. This is similar to our work, but we improve on their work in three different aspects.…”

Section: Related Workmentioning

confidence: 99%

Natural Language Grounding and Grammar Induction for Robotic Manipulation Commands

Alomari¹,

Duckworth²,

Hawasly³

et al. 2017

Proceedings of the First Workshop on Language Grounding for Robotics

View full text Add to dashboard Cite

We present a cognitively plausible system capable of acquiring knowledge in language and vision from pairs of short video clips and linguistic descriptions. The aim of this work is to teach a robot manipulator how to execute natural language commands by demonstration. This is achieved by first learning a set of visual 'concepts' that abstract the visual feature spaces into concepts that have human-level meaning. Second, learning the mapping/grounding between words and the extracted visual concepts. Third, inducing grammar rules via a semantic representation known as Robot Control Language (RCL). We evaluate our approach against state-of-the-art supervised and unsupervised grounding and grammar induction systems, and show that a robot can learn to execute never seenbefore commands from pairs of unlabelled linguistic and visual inputs.

show abstract

Back to the Blocks World: Learning New Actions through Situated Human-Robot Dialogue

Cited by 58 publications

References 15 publications

Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction

Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction

A Natural Language Control System for Application Specific Robots

Natural Language Grounding and Grammar Induction for Robotic Manipulation Commands

Contact Info

Product

Resources

About