Demonstration-Guided Deep Reinforcement Learning of Control Policies for Dexterous Human-Robot Interaction

Abstract

In this paper, we propose a method for training control policies for human-robot interactions such as handshakes or hand claps via Deep Reinforcement Learning. The policy controls a humanoid Shadow Dexterous Hand, attached to a robot arm. We propose a parameterizable multi-objective reward function that allows learning of a variety of interactions without changing the reward structure. The parameters of the reward function are estimated directly from motion capture data of human-human interactions in order to produce policies that are perceived as being natural and human-like by observers. We evaluate our method on three significantly different hand interactions: handshake, hand clap and finger touch. We provide detailed analysis of the proposed reward function and the resulting policies and conduct a large-scale user study, indicating that our policy produces natural looking motions.

Human-Robot Handshaking: A Review

“…Their framework estimates the speed of the interaction as well, to match the speed of the human. Christen et al [17] use Deep Reinforcement Learning (RL) to learn physical interactions from human-human interactions. They use an imitation reward which helps in learning the intricacies of the interaction.…”

Improved Learning of Robot Manipulation Tasks Via Tactile Intrinsic Motivation

“…Our main insight is that sense of touch can guide robots to take actions that result in manipulation of an object. To implement the sense of touch, we extend our state-space with force measurements from tactile sensors positioned at the endeffector [13], [14], [15]. We introduce a touch-based intrinsic reward function to direct exploration towards the states where the robot touches the object.…”

“…More similar to our method, [15] and [18] extend their state space with tactile sensing and use it in the reward function. To learn human-robot interactions, [15] propose a specific reward to ensure contact between the human and the robot. [18] uses the feedback of tactile sensors as a penalty to avoid high impacts and hence to learn gentle manipulation.…”

“…We extend our state space with force measurements from tactile sensors positioned at the end-effector [13], [14], [15]. The original state space consists of a mixture between proprioceptive state information, i.e., joint positions and joint velocities, and relevant information about the manipulation object, such as the object's velocity and the position x obj , which we will use in the reward function.…”

“…In our method (see Figure 1), we add force sensors to the end-effector and extend the state space with force measurements, similarly to other work [13], [14], [15]. We then go beyond prior work and introduce an intrinsic reward to guide the exploration.…”