Following directions using statistical machine translation

Matuszek,; Fox,; Koscher,

doi:10.1109/hri.2010.5453189

Cited by 73 publications

(73 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [28], reinforcement learning is applied to interpret NL directions in terms of landmarks on a map. In [29], machine translation is used to translate from NL route instructions to a map of an environment built by a robot. In [30], Generalized Grounding Graphs are presented that define a probabilistic graphical model dynamically according to linguistic parse structures.…”

Section: Methodsmentioning

confidence: 99%

Multi-purpose natural language understanding linked to sensorimotor experience in humanoid robots

Ovchinnikova

Wächter

Wittenbeck

et al. 2015

2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids)

View full text Add to dashboard Cite

Humans have an amazing ability to bootstrap new knowledge. The concept of structural bootstrapping refers to mechanisms relying on prior knowledge, sensorimotor experience, and inference that can be implemented in robotic systems and employed to speed up learning and problem solving in new environments. In this context, the interplay between the symbolic encoding of the sensorimotor information, prior knowledge, planning, and natural language understanding plays a significant role. In this paper, we show how the symbolic descriptions of the world can be generated on the fly from the continuous robot's memory. We also introduce a multi-purpose natural language understanding framework that processes human spoken utterances and generates planner goals as well as symbolic descriptions of the world and human actions. Both components were tested on the humanoid robot ARMAR-III in a scenario requiring planning and plan recognition based on human-robot communication.

show abstract

Section: Methodsmentioning

confidence: 99%

Multi-purpose natural language understanding linked to sensorimotor experience in humanoid robots

Ovchinnikova

Wächter

Wittenbeck

et al. 2015

2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids)

View full text Add to dashboard Cite

show abstract

“…searches semantic maps for action sequences that maximize the metric of words and graphs. Matuszek, Fox, and Koscher [10] map utterances to a path description language to follow directions in a labeled map. Zender et al [11] builds a navigation roadmap of admissible states from sensor data and a natural language dialog system.…”

Section: Related Workmentioning

confidence: 99%

A natural language planner interface for mobile manipulators

Howard

Tellex

Roy

2014

2014 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Natural language interfaces for robot control aspire to find the best sequence of actions that reflect the behavior intended by the instruction. This is difficult because of the diversity of language, variety of environments, and heterogeneity of tasks. Previous work has demonstrated that probabilistic graphical models constructed from the parse structure of natural language can be used to identify motions that most closely resemble verb phrases. Such approaches however quickly succumb to computational bottlenecks imposed by construction and search the space of possible actions. Planning constraints, which define goal regions and separate the admissible and inadmissible states in an environment model, provide an interesting alternative to represent the meaning of verb phrases. In this paper we present a new model called the Distributed Correspondence Graph (DCG) to infer the most likely set of planning constraints from natural language instructions. A trajectory planner then uses these planning constraints to find a sequence of actions that resemble the instruction. Separating the problem of identifying the action encoded by the language into individual steps of planning constraint inference and motion planning enables us to avoid computational costs associated with generation and evaluation of many trajectories. We present experimental results from comparative experiments that demonstrate improvements in efficiency in natural language understanding without loss of accuracy.

show abstract

“…It has been used for instance for natural language processing for applications such as direction recognition [9], [10] or language grounding [11]. [12] presented a spatial reasoner integrated in a robot which computes symbolic positions of objects We use perspective taking and some elements of theory of mind techniques to efficiently compute perspective-aware models of the world.…”

Section: Related Workmentioning

confidence: 99%

A framework for endowing an interactive robot with reasoning capabilities about perspective-taking and belief management

Milliez

Warnier

Clodic

et al. 2014

The 23rd IEEE International Symposium on Robot and Human Interactive Communication

View full text Add to dashboard Cite

International audienceIn daily human interactions, spatial reasoning occupies an important place. In this paper we present a situation assessment reasoner that generates relevant symbolic information from the geometry of the environment with respect to relations between objects and human capabilities. The role of SPARK (SPAtial Reasoning and Knowledge) component is to permanently maintain a state of the world in order to provide a basis for the robot to plan, to act, to react and to interact. More precisely, we describe here the way the system manages the hypotheses to be able to handle such knowledge in a flexible manner. Equipped with such capabilities, a robot that will interact with humans should be able to extract, compute or infer these relations and capabilities in order to communicate and interact efficiently in a natural way. To illustrate our work, we will explain how the robot is able to manage and update agents beliefs and pass Sally-Anne test. This work is part of a broader effort to develop a complete decisional framework for human-robot interactive task achievement

show abstract

Following directions using statistical machine translation

Cited by 73 publications

References 15 publications

Multi-purpose natural language understanding linked to sensorimotor experience in humanoid robots

Multi-purpose natural language understanding linked to sensorimotor experience in humanoid robots

A natural language planner interface for mobile manipulators

A framework for endowing an interactive robot with reasoning capabilities about perspective-taking and belief management

Contact Info

Product

Resources

About