Real-World Human-Robot Collaborative Reinforcement Learning

Abstract

The intuitive collaboration of humans and intelligent robots (embodied AI) in the real-world is an essential objective for many desirable applications of robotics. Whilst there is much research regarding explicit communication, we focus on how humans and robots interact implicitly, on motor adaptation level. We present a real-world setup of a humanrobot collaborative maze game, designed to be non-trivial and only solvable through collaboration, by limiting the actions to rotations of two orthogonal axes, and assigning each axes to one player. This results in neither the human nor the agent being able to solve the game on their own. We use a stateof-the-art reinforcement learning algorithm for the robotic agent, and achieve results within 30 minutes of real-world play, without any type of pre-training. We then use this system to perform systematic experiments on human/agent behaviour and adaptation when co-learning a policy for the collaborative game. We present results on how co-policy learning occurs over time between the human and the robotic agent resulting in each participant's agent serving as a representation of how they would play the game. This allows us to relate a person's success when playing with different agents than their own, by comparing the policy of the agent with that of their own agent.

The effects of role transitions and adaptation in human–cobot collaboration

“…Nikolaidis et al [33] introduced a formalization for mutual adaptation between a robot and a human in a collaborative task. In a similar way, the study in [19] present a reinforcement learning algorithm able to solve human-robot task in which neither the human nor the robot is able to solve the problem on their own. Ikemoto et al [34] showed the importance of a bilateral learning process that takes place in both partners.…”

“…For instance, Peternel et al [18] proposed a method for human-robot collaboration where the robot behavior is adapted online to the human motor fatigue. In other situations, adaptation can be used to solve problems that in which neither the human nor the robot is able to solve the problem on their own [19].…”