PVRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction

Abstract

Human motion prediction, which aims to predict future human poses given past poses, has recently seen increased interest. Many recent approaches are based on Recurrent Neural Networks (RNN) which model human poses with exponential maps. These approaches neglect the pose velocity as well as temporal relation of different poses, and tend to converge to the mean pose or fail to generate naturallooking poses. We therefore propose a novel Position-Velocity Recurrent Encoder-Decoder (PVRED) for human motion prediction, which makes full use of pose velocities and temporal positional information. A temporal position embedding method is presented and a Position-Velocity RNN (PVRNN) is proposed. We also emphasize the benefits of quaternion parameterization of poses and design a novel trainable Quaternion Transformation (QT) layer, which is combined with a robust loss function during training. Experiments on two human motion prediction benchmarks show that our approach considerably outperforms the state-of-the-art methods for both short-term prediction and long-term prediction. In particular, our proposed approach can predict future human-like and meaningful poses in 4000 milliseconds.

Continuous Decoding of Daily-Life Hand Movements from Forearm Muscle Activity for Enhanced Myoelectric Control of Hand Prostheses

“…To learn the mapping between the complete feature set and hand joint accelerations, we used LSTM-based networks [28]. This choice was motivated by previous work, which showed that the ability of gated recurrent neural networks to learn long-term relationships between signals is instrumental in allowing the prediction of whole-body human motion [37,38]. Moreover, LSTM-based networks have recently been proved to be effective at predicting wrist position (3 DOFs) [39] and wrist flexion/extension (1 DOFs) [40] by tracking the EMG activity of 2-5 shoulder and arm muscles.…”

Hierarchical Human-Motion Prediction and Logic-Geometric Programming for Minimal Interference Human-Robot Tasks

“…Nonlinear function approximators such as Gaussian Processes [24] or Deep Neural Networks [9], [25] have been used to regress large databases of human movement. Recurrent Neural Networks (RNN) are state of the art for predicting short-term high dimensional movements [8], [9], [26].…”

“…Thus, we need a predictive function h t+1:T = f (h 0:t , g * ) that computes a trajectory of future human states h t+1:T given previous observed states h 0:t and a goal g * . We use VRED, a recurrent neural network-based model for predicting motion [8] and make it goal-conditioned by adding a three-dimensional position g t to the input of the network at every timestep (see Figure 2). The goal input g t is relative to the coordinate frame of the human and thus changes every timestep.…”

“…As human motion is the result of complex biomechanical processes that are challenging to model, state-of-the-art work on motion prediction focuses on data-driven models, such as recurrent neural networks [8], [9], [10]. Our prediction approach makes use of Maximum Entropy Inverse Reinforcement Learning [11], to produce a discrete policy, this policy decides what actions to take next based on the symbolic states of the world.…”

“…At the lowest level we predict movements using a Recurrent Neural Network (RNN), which is trained on the MoGaze dataset [2]. In order to adapt the discrete policy to continuous motion prediction, we introduce goal conditioning to the RNN VRED architecture [8]. We combine two networks, one conditioned on hand target goals for manipulation and another one on pelvis target goals for walking.…”