Learning optimal navigation actions for foresighted robot behavior during assistance tasks

Abstract

We present an approach to learn optimal navigation actions for assistance tasks in which the robot aims at efficiently reaching the final navigation goal of a human where service has to be provided. Always following the human at a close distance might hereby result in inefficient trajectories, since people regularly do not move on the shortest path to their destination (e.g., they move to grab the phone or make a note). Therefore, a service robot should infer the human's intended navigation goal and compute its own motion based on that prediction. We developed an approach that applies reinforcement learning to get a Q-function that determines for each pair of the robot's and human's relative positions the best navigation action for the robot. Our approach applies a prediction of the human's motion based on a softened Markov decision process (MDP). This MDP is independent from the navigation learning framework and is learned beforehand on previously observed trajectories. We thoroughly evaluated our method in simulation and on a real robot. As the experimental results show, our approach leads to foresighted navigation behavior and significantly reduces the path length and completion time compared to naive following strategies.

Learning Personalized Human-Aware Robot Navigation Using Virtual Reality Demonstrations from a User Study

“…Methods like SARL [36] or MP-RGL [37] capture interactions between robot and humans with excellent results, but rely on the known velocity of humans. Others infer or forecast human behavior by predicting their longterm goals, or by predicting their future motion and activities [38], often by employing 3D lidar or RGB(D) cameras [39]- [43]. Wang et al [44] processed optical flow from RGB data as temporal information with an attention module to detect actions.…”

Toward stable astronaut following of extravehicular activity assistant robots using deep reinforcement learning

“…Some innovative attempts have been made to improve the intelligence and efficiency of the following robot, given knowledge of the target human's predicted movements. 19,20 For example, Bayoumi and Bennewitz predicted human motion using a modified Markov decision process (MDP) 21 and developed a DRL-based navigator, which resulted in foresighted following behaviors and shorter following trajectories. 19 Recovery mechanisms, based on human trajectory prediction, have also been implemented when the robot misses a target in dynamic environments.…”

“…19,20 For example, Bayoumi and Bennewitz predicted human motion using a modified Markov decision process (MDP) 21 and developed a DRL-based navigator, which resulted in foresighted following behaviors and shorter following trajectories. 19 Recovery mechanisms, based on human trajectory prediction, have also been implemented when the robot misses a target in dynamic environments. 20 These studies suggested that the prediction of user movement could significantly improve following efficiency and the foresightedness of the robot, but the mechanisms how it contributes to following stability remain to be addressed.…”