Learning Dynamic Robot-to-Human Object Handover from Human Feedback

Abstract

Object handover is a basic, but essential capability for robots interacting with humans in many applications, e.g., caring for the elderly and assisting workers in manufacturing workshops. It appears deceptively simple, as humans perform object handover almost flawlessly. The success of humans, however, belies the complexity of object handover as collaborative physical interaction between two agents with limited communication. This paper presents a learning algorithm for dynamic object handover, for example, when a robot hands over water bottles to marathon runners passing by the water station. We formulate the problem as contextual policy search, in which the robot learns object handover by interacting with the human. A key challenge here is to learn the latent reward of the handover task under noisy human feedback. Preliminary experiments show that the robot learns to hand over a water bottle naturally and that it adapts to the dynamics of human motion. One challenge for the future is to combine the model-free learning algorithm with a model-based planning approach and enable the robot to adapt over human preferences and object characteristics, such as shape, weight, and surface texture.

Probabilistic movement primitives for coordination of multiple human–robot collaborative tasks

“…As previously investigated (Strabala et al, 2013), these steps range from (1) the social-cognitive cues that establish the connection between the giver and the taker, (2) the coordination of the location and the resulting trajectory as a function of preferences and socially acceptable movements (Sisbot & Alami, 2012), and (3) the final physical transfer that comprises interaction forces and compliances (Kupcsik, Hsu, & Lee, 2015).…”

Autonomous Object Handover Using Wrist Tactile Information

“…The mean value of relative standard deviation for force magnitude is just 0.03, while the same value for the derivative of force (jerk) is 17. 6. This means that the use of jerk can provide a better estimate of the time instant when the contact occurs.…”

“…Generally, handover is performed based on human motion patterns that are studied and implemented on the robot [5], or they are used as input to a learning-bydemonstration system [6]. Whether the human action is learned or used as a reference, the existing approaches are computationally complex, require online robotic learning or a large data set of demonstrations.…”

Interactive Trajectory Adaptation through Force-guided Bayesian Optimization

“…expensive trials, unknown or hard-to-model objective function and data efficiency). BayesOpt has recently gained interest in different robotic applications, such as behavior adaptation for damaged robots [8], automatic controller tuning for balancing [9], locomotion [10], and interaction tasks [11], and in physical HRC [12], [13]. BayesOpt was used in [12] to select the collaborative actions that minimize a Q-function in a model-based reinforcement learning approach.…”

“…Interaction forces and positional data, representing the robot state, were exploited to determine the exploration actions. In [13], the authors employed BayesOpt to approximate an unknown reward function for a contextual policy search aimed at learning a handover skill by interacting with a human. However, both [12], [13] did not exploit the fact that the robot may learn an initial policy from demonstrations, which may speed up the learning process and reduce explorations.…”