Learning human-robot handovers through π-STAM: Policy improvement with spatio-temporal affordance maps

Abstract

Human-robot handovers are characterized by high uncertainty and poor structure of the problem that make them difficult tasks. While machine learning methods have shown promising results, their application to problems with large state dimensionality, such as in the case of humanoid robots, is still limited. Additionally, by using these methods and during the interaction with the human operator, no guarantees can be obtained on the correct interpretation of spatial constraints (e.g., from social rules). In this paper, we present Policy Improvement with Spatio-Temporal Affordance Mapsπ-STAM, a novel iterative algorithm to learn spatial affordances and generate robot behaviors. Our goal consists in generating a policy that adapts to the unknown action semantics by using affordances. In this way, while learning to perform a human-robot handover task, we can (1) efficiently generate good policies with few training episodes, and (2) easily encode action semantics and, if available, enforce prior knowledge in it. We experimentally validate our approach both in simulation and on a real NAO robot whose task consists in taking an object from the hands of a human. The obtained results show that our algorithm obtains a good policy while reducing the computational load and time duration of the learning process.

10 Years of Human-NAO Interaction Research: A Scoping Review

“…Both studies suggest that a robot's presence did not affect the degree of trustworthiness and appraisal, and user enjoyment, but the perceived level of robot intelligence may decrease when people know about teleoperation. Some studies explored robot personality effect on interaction quality such as extroverted vs. introverted (Aly and Tapus, 2013;Celiktutan and Gunes, 2015), low interactive vs. high interactive (Tozadore et al, 2016;Horstmann and Krämer, 2020), active vs. passive (Mubin et al, 2014), affective vs. non-affective (Tielman et al, 2014), emotional vs. unemotional and high vs. low intelligence (Zlotowski et al, 2014), lack of ability vs. lack of effort (van der Woerdt and Haselager, 2019), and simulated vs. real robot (Riccio et al, 2016). The robot-to-robot interaction and comparisons were also carried out in different contexts.…”

Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics

“…Machine learning methods are increasingly used in robotic applications due to advances in their mathematical formalization. Deep learning and reinforcement learning are among the most promising methods applied in collaborative robotics, with their own levels of maturity and different challenges for real-world applications [137][138][139][140][141][142].…”

Evaluating Guided Policy Search for Human-Robot Handovers

“…Other approaches have used dynamical systems [20], look-up tables [21], or neural networks [22], [23] to encode the demonstrations and generate robot motion in the reach phase. Some researchers have used reinforcement learning techniques to learn online controllers for the reach phase from human feedback [18], [19].…”