Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors

Abstract

We investigate the use of prior knowledge of human and animal movement to learn reusable locomotion skills for real legged robots. Our approach builds upon previous work on imitating human or dog Motion Capture (MoCap) data to learn a movement skill module. Once learned, this skill module can be reused for complex downstream tasks. Importantly, due to the prior imposed by the MoCap data, our approach does not require extensive reward engineering to produce sensible and natural looking behavior at the time of reuse. This makes it easy to create well-regularized, task-oriented controllers that are suitable for deployment on real robots. We demonstrate how our skill module can be used for imitation, and train controllable walking and ball dribbling policies for both the ANYmal quadruped and OP3 humanoid. These policies are then deployed on hardware via zero-shot simulation-to-reality transfer. Accompanying videos are available at https://bit.ly/robot-npmp.

Versatile multicontact planning and control for legged loco-manipulation

“…However, without a tedious reward-engineering process dedicated to the task at hand, it generally struggles to discover the right behaviors. This can be addressed by leveraging expert demonstrations: Multimodal trajectories can be encoded into motion priors that are then used to guide the training of a robust RL agent (39,81,82) in complex loco-manipulation settings. However, instead of relying on expert trajectories generated by human examples (83,84), one could potentially use our framework as an automatic provider of physically consistent demonstrations for an RL pipeline.…”

“…One common way of overcoming this issue has been by bootstrapping RL with a form of imitation learning. For example, training can be guided by recorded human demonstrations (28,29), animal motion capture clips (38,39), or an RL-trained teacher with access to privileged information (27,40,41). Nevertheless, generating expert demonstrations for every newly encountered task is time consuming, and motion retargeting is often challenging, especially when the robot's morphology differs from that of the demonstrator.…”

Learning Agile, Robust Locomotion Skills for Quadruped Robot

“…[13] revisited the necessity of dynamics randomization in legged locomotion and gave suggestions on where and how to use dynamics randomization. [14] trained a low-level quadruped robot locomotion controller by imitating real animal data and used the low-level controller to accomplish different tasks. [15] trained policies to jump from pixel inputs.…”

DTC: Deep Tracking Control

“…A notable line of research has focused on learning a specific task while imitating expert behavior. The expert provides a direct demonstration for solving the task (22,23) or is used to impose a style while discovering the task (24)(25)(26). These approaches require collecting expert data, commonly done offline, through either retargeted motion capture data (24)(25)(26) or a TO technique (22,23).…”

“…The expert provides a direct demonstration for solving the task (22,23) or is used to impose a style while discovering the task (24)(25)(26). These approaches require collecting expert data, commonly done offline, through either retargeted motion capture data (24)(25)(26) or a TO technique (22,23). The reward function can then be formulated to be dense, meaning that agents can collect nontrivial rewards even if they do not initially solve the task.…”