Mode-adaptive neural networks for quadruped motion control

Abstract

Quadruped motion includes a wide variation of gaits such as walk, pace, trot and canter, and actions such as jumping, sitting, turning and idling. Applying existing data-driven character control frameworks to such data requires a significant amount of data preprocessing such as motion labeling and alignment. In this paper, we propose a novel neural network architecture called Mode-Adaptive Neural Networks for controlling quadruped characters. The system is composed of the motion prediction network and the gating network. At each frame, the motion prediction network computes the character state in the current frame given the state in the previous frame and the user-provided control signals. The gating network dynamically updates the weights of the motion prediction network by selecting and blending what we call the expert weights, each of which specializes in a particular movement. Due to the increased flexibility, the system can learn consistent expert weights across a wide range of non-periodic/periodic actions, from unstructured motion capture data, in an end-to-end fashion. In addition, the users are released from performing complex labeling of phases in different gaits. We show that this architecture is suitable for encoding the multi-modality of quadruped locomotion and synthesizing responsive motion in real-time.

Interactive Learning of Temporal Features for Control: Shaping Policies and State Representations From Human Feedback

“…The previously discussed shortcoming of DAgger algorithms opens possibilities for future works intended to study how to deal with databases that have mistaken examples. Another field of study is data-efficient movement generation in animation [19], which, combined with our method, would make it possible to learn (non)periodic movements using spatiotemporal features and IIL. Challenges such as the generation of smooth, precise, and stylistic movements (i.e., dealing with high-frequency details [20]) could be also addressed.…”

Analyzing Input and Output Representations for Speech-Driven Gesture Generation

“…Generating body motion is an active area of research with applications to animation, computer games, and other simulations. Current state-of-the-art approaches in such body-motion generation are generally data-driven and based on deep learning [28,40,41]. Zhou et al [41] proposed a modified training regime to make recurrent neural networks generate human motion with greater long-term stability, while Pavllo et al [28] formulated separate short-term and long-term recurrent motion predictors, using quaternions to more adequately express body rotations.…”

Automatic Sign Dance Synthesis from Gesture-based Sign Language

“…Holden et al generated realistic, controllable locomotion by considering the different stepping phase of a locomotion patter. Zhang et al [35] synthesized quadruped motion by combining a motion prediction network that computes the character state, and a gating network that dynamically updates the weights of the motion prediction network. Holden et al [14] adapted an autoencoder with a convolutional neural network focusing on the temporal aspect of human motion to generate a effective latent space for synthesizing new motion.…”