Learning sequential tasks interactively from demonstrations and own experience

Abstract

Abstract-Deploying robots to our day-to-day life requires them to have the ability to learn from their environment in order to acquire new task knowledge and to flexibly adapt existing skills to various situations. For typical real-world tasks, it is not sufficient to endow robots with a set of primitive actions. Rather, they need to learn how to sequence these in order to achieve a desired effect on their environment. In this paper, we propose an intuitive learning method for a robot to acquire sequences of motions by combining learning from human demonstrations and reinforcement learning. In every situation, our approach treats both ways of learning as alternative control flows to optimally exploit their strengths without inheriting their shortcomings. Using a Gaussian Process approximation of the state-action sequence value function, our approach generalizes values observed from demonstrated and autonomously generated action sequences to unknown inputs. This approximation is based on a kernel we designed to account for different representations of tasks and action sequences as well as inputs of variable length. From the expected deviation of value estimates, we devise a greedy exploration policy following a Bayesian optimization criterion that quickly converges learning to promising action sequences while protecting the robot from sequences with unpredictable outcome. We demonstrate the ability of our approach to efficiently learn appropriate action sequences in various situations on a manipulation task involving stacked boxes.

Learning and Sequencing of Object-Centric Manipulation Skills for Industrial Tasks

“…Given a set of skills, the next problem is how to combine them for successful execution of complex manipulation tasks. Researchers have mainly used either LfD [20], [21], [22] or reinforcement learning (RL) [23], [24] to master skills sequencing, most of them employing dynamic movement primitives (DMPs) as skill representation. Manschitz et al [20] learn a sequence graph of skills from kinesthetic demonstrations, where a classifier drives transitions between skills.…”

“…Gräve and Behnke [23] combine LfD and RL to learn a sequence of skills in an active learning setting. In [24], a modified PI 2 method adapts the shape and attractor of several learned DMPs to smoothly sequence them.…”