Interactive Robot Training for Temporal Tasks

Abstract

Defining sound and complete specifications for robots using formal languages is challenging, while learning formal specifications directly from demonstrations can lead to over-constrained task policies. In this paper, we propose a Bayesian interactive robot training framework that allows the robot to learn from both demonstrations provided by a teacher, and that teacher's assessments of the robot's task executions. We also present an active learning approach -inspired by uncertainty sampling -to identify the task execution with the most uncertain degree of acceptability. We demonstrate that active learning within our framework identifies a teacher's intended task specification to a greater degree of similarity when compared with an approach that learns purely from demonstrations. Finally, we also conduct a user-study that demonstrates the efficacy of our active learning framework in learning a table-setting task from a human teacher.

Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences

“…Prior work has extensively studied learning reward functions using a single source of information, e.g., ordinal data (Chu and Ghahramani, 2005) or human corrections (Bajcsy et al, 2018(Bajcsy et al, , 2017Li et al, 2021b). Other works attempted to incorporate expert assessments of trajectories (Shah and Shah, 2020). More related to our work, we focus on learning from demonstrations and learning from rankings.…”