Grounding English Commands to Reward Functions

Abstract

Abstract-As intelligent robots become more prevalent, methods to make interaction with the robots more accessible are increasingly important. Communicating the tasks that a person wants the robot to carry out via natural language, and training the robot to ground the natural language through demonstration, are especially appealing approaches for interaction, since they do not require a technical background. However, existing approaches map natural language commands to robot command languages that directly express the sequence of actions the robot should execute. This sequence is often specific to a particular situation and does not generalize to new situations. To address this problem, we present a system that grounds natural language commands into reward functions using demonstrations of different natural language commands being carried out in the environment. Because language is grounded to reward functions, rather than explicit actions that the robot can perform, commands can be high-level, carried out in novel environments autonomously, and even transferred to other robots with different action spaces. We demonstrate that our learned model can be both generalized to novel environments and transferred to a robot with a different action space than the action space used during training.

Sequence-to-Sequence Language Grounding of Non-Markovian Task Specifications

“…The question of how to effectively convert between natural language instructions and robot behavior has been widely studied in previous work [50,34,24,14,9,47,8,18,11,1,33,28,36,27,40,7,2,19,37]. So far, there have been three categories of behavior specifications that these works have mapped natural language to: action sequences, goal states, and LTL specifications.…”

“…Following the framework introduced by MacGlashan et al [33], we treat natural language as the specification of a latent reward function that completes the definition of an otherwise fully-specified MDP. We use a language grounding model to arrive at a more consolidated, semantic representation of that reward function, thereby completing the MDP and allowing it to be passed to an arbitrary planning algorithm for generating robot behavior.…”

“…The OO-MDP propositional functions serve as possible atoms of the LTL expressions, creating an expressive language for defining tasks. In particular, LTL formulae are sufficiently expressive to subsume semantic representations used in previous goal-based language grounding work such as MacGlashan et al [33] and Arumugam et al [2].…”

Proceedings of the First Workshop on Language Grounding for Robotics

“…Here, we learn an adaptive strategy that aims at maximising the overall learning performance simultaneously, by properly adjusting the positive confidence threshold in the range of 0.65 to 0.95. We train the optimization using a RL libraryBurlap (MacGlashan, 2015) as follows, in detail:…”

“…There has been a broad and diverse set of work examining how best to interpret and execute natural language instructions on a robot platform (Vogel and Jurafsky, 2010;Tellex et al, 2011;Artzi and Zettlemoyer, 2013;Howard et al, 2014;Andreas and Klein, 2015;MacGlashan et al, 2015;Paul et al, 2016;Mei et al, 2016;Arumugam et al, 2017). Vogel and Jurafsky (2010) produce policies using language and expert trajectories based rewards, which allow for planning within a stochastic environment along with re-planning in case of failure.…”