MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control

Abstract

Simulated humanoids are an appealing research domain due to their physical capabilities. Nonetheless, they are also challenging to control, as a policy must drive an unstable, discontinuous, and high-dimensional physical system. One widely studied approach is to utilize motion capture (MoCap) data to teach the humanoid agent low-level skills (e.g., standing, walking, and running) that can then be re-used to synthesize high-level behaviors. However, even with MoCap data, controlling simulated humanoids remains very hard, as MoCap data offers only kinematic information. Finding physical control inputs to realize the demonstrated motions requires computationally intensive methods like reinforcement learning. Thus, despite the publicly available MoCap data, its utility has been limited to institutions with large-scale compute. In this work, we dramatically lower the barrier for productive research on this topic by training and releasing high-quality agents that can track over three hours of MoCap data for a simulated humanoid in the dm_control physics-based environment. We release MoCapAct, a dataset of these expert agents and their rollouts, which contain proprioceptive observations and actions. We demonstrate the utility of MoCapAct by using it to train a single hierarchical policy capable of tracking the entire MoCap dataset within dm_control and show the learned low-level component can be re-used to efficiently learn downstream high-level tasks. Finally, we use MoCapAct to train an autoregressive GPT model and show that it can control a simulated humanoid to perform natural motion completion given a motion prompt. Videos of the results and links to the code and dataset are available at the project website.

SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation

“…Distillation has been applied in previous efforts to scale up reinforcement learning to train multi-task control policies [Merel et al 2019[Merel et al , 2020. Wagener et al [2023] used a single-stage distillation approach to train a general motion tracking controller capable of imitating approximately 3.5 hours of motion data. Our multi-stage progress distillation approach enables our system to train versatile controllers on 8.5 hours of text-labeled motion clips, leading to a unified end-to-end controller that can be directed to perform a large variety skills with simple text commands.…”

“…Our approach is inspired by the first stage of MoCapAct [Wagener et al 2023], leveraging DeepMimic as our tracking method [Peng et al 2018]. We train a DeepMimic expert policy 𝜋 𝑒 𝑖 (a 𝑡 |o 𝑡 , 𝜙) on every motion capture sequence in our dataset, conditioned on the current state of character o 𝑡 as well as a phase variable 𝜙 ∈ [0, 1] that synchronizes the policy to the reference motion.…”