Interactive robot task training through dialog and demonstration

Abstract

Effective human/robot interfaces which mimic how humans interact with one another could ultimately lead to robots being accepted in a wider domain of applications. We present a framework for interactive task training of a mobile robot where the robot learns how to do various tasks while observing a human. In addition to observation, the robot listens to the human's speech and interprets the speech as behaviors that are required to be executed. This is especially important where individual steps of a given task may have contingencies that have to be dealt with depending on the situation. Finally, the context of the location where the task takes place and the people present factor heavily into the robot's interpretation of how to execute the task. In this paper, we describe the task training framework, describe how environmental context and communicative dialog with the human help the robot learn the task, and illustrate the utility of this approach with several experimental case studies.

Graph-Based Task Libraries for Robots: Generalization and Autocompletion

“…Approaches focus on teaching a single tasks, using varying representations. Tasks have been represented as acyclic graphs composed of nodes representing finite state machines [21] without loops or variables. More recently, complex tasks are represented as Instruction Graphs [15], which only handle instantiated tasks, or Petri Net Plans [8], which support parameterized tasks defined by a user.…”

“…For a more detailed overview of Instruction Graphs, we refer the reader to [15]. However, the generalization algorithms we discuss can be applied to other graph-like representations [8,21].…”

Engaging human-to-robot attention using conversational gestures and lip-synchronization

“…This is done by using technologies such as audio synthesizers (TTS) [17] and speech recognition systems (ASR) [18]. In the last years, these systems are becoming very common in social robotics [19], [20], [21], [22].…”

Mobile Robot Motion Control from Demonstration and Corrective Feedback

“…Successful LfD applications under this ap-proach span tasks with autonomous helicopters [1,9,27] to small quadriped robots [22,32], humanoids [24] and robotic arms [8]. The third category learns task plans from the demonstration set, including small wheeled robot [29] and companion robot [33] applications. Note that each of these techniques are employed for the derivation of policies with high-level actions, while only the first two are used to derive policies with low-level actions.…”