Robust spoken instruction understanding for HRI

Cantrell, Rehj; Scheutz, Matthias; Schermerhorn, Paul; Wu, Xuan

doi:10.1145/1734454.1734555

Cited by 26 publications

(27 citation statements)

References 6 publications

(6 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Others have created robotic systems that interact using dialog [5,15,2]. Bauer et al [1] built a robot that can find its way through an urban environment by interacting with pedestrians using a touch screen and gesture recognition system.…”

Section: B Asking Questionsmentioning

confidence: 99%

Toward Information Theoretic Human-Robot Dialog

Tellex¹,

Thaker²,

Deits³

et al. 2012

Robotics: Science and Systems VIII

View full text Add to dashboard Cite

Abstract-Our goal is to build robots that can robustly interact with humans using natural language. This problem is challenging because human language is filled with ambiguity, and furthermore, due to limitations in sensing, the robot's perception of its environment might be much more limited than that of its human partner. To enable a robot to recover from a failure to understand a natural language utterance, this paper describes an information-theoretic strategy for asking targeted clarifying questions and using information from the answer to disambiguate the language. To identify good questions, we derive an estimate of the robot's uncertainty about the mapping between specific phrases in the language and aspects of the external world. This metric enables the robot to ask a targeted question about the parts of the language for which it is most uncertain. After receiving an answer, the robot fuses information from the command, the question, and the answer in a joint probabilistic graphical model in the G 3 framework. When using answers to questions, we show the robot is able to infer mappings between parts of the language and concrete object groundings in the external world with higher accuracy than by using information from the command alone. Furthermore, we demonstrate that by effectively selecting which questions to ask, the robot is able to achieve significant performance gains while asking many fewer questions than baseline metrics.

show abstract

Section: B Asking Questionsmentioning

confidence: 99%

Toward Information Theoretic Human-Robot Dialog

Tellex¹,

Thaker²,

Deits³

et al. 2012

Robotics: Science and Systems VIII

View full text Add to dashboard Cite

show abstract

“…This makes language disfluent and requires repair. For example, "There is a blue kit, erm, I mean a blue box, at the end of the table" can be simplified to "There is a blue box at the end of the table" [11]. f) Indirect assertions through relative clauses and appositions: Relative clauses and appositions are often used for indirect assertions of world properties.…”

Section: Preliminaries and Related Work A What Makes Natural Lanmentioning

confidence: 99%

Exploiting deep semantics and compositionality of natural language for Human-Robot-Interaction

Eppe

Trott

Feldman

2016

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

We develop a natural language interface for human robot interaction that implements reasoning about deep semantics in natural language. To realize the required deep analysis, we employ methods from cognitive linguistics, namely the modular and compositional framework of Embodied Construction Grammar (ECG) [16]. Using ECG, robots are able to solve finegrained reference resolution problems and other issues related to deep semantics and compositionality of natural language. This also includes verbal interaction with humans to clarify commands and queries that are too ambiguous to be executed safely. We implement our NLU framework as a ROS package and present proof-of-concept scenarios with different robots, as well as a survey on the state of the art.

show abstract

“…Repeated -conversions then lead to -free temporal and dynamic formulas that represent the goals and actions specified in the natural language instruction, respectively. We also have developed algorithms for handling disfluencies, in particular, lexical disfluencies, abandoned utterances, repetitions, as well as some repairs and corrections, in the context of spoken instruction understanding (Cantrell et al 2010). Much of this work was directly based on our CReST corpus.…”

Section: Human Performance In a Search Taskmentioning

confidence: 99%

Toward Humanlike Task‐Based Dialogue Processing for Human Robot Interaction

2011

Self Cite

View full text Add to dashboard Cite

ArticlesWINTER 2011 77 I nteractions in natural language dialogues are an essential part of human social exchanges, ranging from social conventions such as greetings, to simple question-answer pairs, to task-based dialogues for coordinating activities, topic-based discussions, and all kinds of more open-ended conversations. As a result, the ability of future social and service robots to interact with humans in natural ways ) will critically depend on developing capabilities of humanlike dialoguebased natural language processing (NLP) in robotic architectures. However, different from other NLP contexts such as story understanding or machine translation, natural language processing on robots has at least the following six properties: realtime, parallel, spoken, embodied, situated, and dialogue-based.Real-time means that all processing must occur within the time frame of human processing, both at the level of comprehension as well as production. It also means that constraints will have to be incorporated incrementally as they occur, analogous to human language processing.Parallel means that all stages of language processing must operate concurrently to mutually constrain possible meaning interpretations and to allow for the generation of responses (such as acknowledgements) while an ongoing utterance is being processed.Spoken means that language processing necessarily operates on imperfect acoustic signals with varying quality that depends on the speaker and the background noise. In addition to handling prosodic variations, this includes typical features of spontaneous speech such as various types of disfluencies, slips of the tongue, or other types of errors that are usually not found in written texts.Embodied means that robots have to be able to process multimodal linguistic cues such as deictic terms accompanied by bodily movements, or other gestures that constrain possible interpretations of linguistic expressions. It also means that the robot will have to be able to produce similar gestures that are expected by human interlocutors to accompany certain linguistic constructs.

show abstract

Robust spoken instruction understanding for HRI

Cited by 26 publications

References 6 publications

Toward Information Theoretic Human-Robot Dialog

Toward Information Theoretic Human-Robot Dialog

Exploiting deep semantics and compositionality of natural language for Human-Robot-Interaction

Toward Humanlike Task‐Based Dialogue Processing for Human Robot Interaction

Contact Info

Product

Resources

About