Modeling Human Motion with Quaternion-Based Neural Networks

Pavllo, Dario; Feichtenhofer, Christoph; Auli, Michael; Grangier, David

doi:10.1007/s11263-019-01245-6

Cited by 149 publications

(189 citation statements)

References 72 publications

Supporting

Mentioning

189

Contrasting

Order By: Relevance

“…We follow the same evaluation setting as in previous work for direct comparability. It is noteworthy to mention that the evaluation metrics reported on H3.6M exhibit high variance due to the small number of test samples [24] and low errors do not always correspond to good qualitative results [20].…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Structured Prediction Helps 3D Human Motion Modelling

Aksan

Kaufmann

Hilliges

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

187

151

View full text Add to dashboard Cite

Figure 1: We introduce a structured prediction layer (SPL) to the task of 3D human motion modelling. The SP-layer explicitly decomposes the pose into individual joints and can be interfaced with a variety of baseline architectures. We show that on H3.6M and a recent, much larger dataset, AMASS, a variety of baseline models benefit when augmented with an SP-layer. AbstractHuman motion prediction is a challenging and important task in many computer vision application domains. Existing work only implicitly models the spatial structure of the human skeleton. In this paper, we propose a novel approach that decomposes the prediction into individual joints by means of a structured prediction layer that explicitly models the joint dependencies. This is implemented via a hierarchy of small-sized neural networks connected analogously to the kinematic chains in the human body as well as a joint-wise decomposition in the loss function. The proposed layer is agnostic to the underlying network and can be used with existing architectures for motion modelling. Prior work typically leverages the H3.6M dataset. We show that some state-of-the-art techniques do not perform well when trained and tested on AMASS, a recently released dataset 14 times the size of H3.6M. Our experiments indicate that the proposed layer increases the performance of motion forecasting irrespective of the base network, jointangle representation, and prediction horizon. We furthermore show that the layer also improves motion predictions qualitatively. We make code and models publicly available at https://ait.ethz.ch/projects/2019/spl.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Input poses are represented as exponential maps. QuaterNet uses a quaternion representation instead [24,25]. The model augments RNNs with quaternion based normalization and regularization operations.…”

Section: Modelsmentioning

confidence: 99%

Structured Prediction Helps 3D Human Motion Modelling

Aksan

Kaufmann

Hilliges

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

187

151

View full text Add to dashboard Cite

show abstract

“…Finally, we choose the 2 Note, that the BPM provided from MSD is computed from BPM estimation algorithm, which well known for its frequent octave error where the erroneous estimates are the integer multiple of its ground-truth value. 3 ConvBlock [8,9,4] ConvBlock [8,9,4] ConvBlock [16,5,2] ConvBlock [16,5,2] gated tanh function as the core non-linearity [14] for convolution blocks and rectified linear unit (ReLU) [20] for later fully connected (FC) layers. The general overview of the architecture can be found in Figure 3.…”

Section: B Music Feature Encodermentioning

confidence: 99%

“…There have been several attempts related to 3D human motion modeling. There exist studies related to 3D joint sequence prediction for the character motion synthesizing such as human locomotion generation [2], [3]. However, these are for generating a motion itself, while our objective is generating a motion dependent to the specific source sequence.…”

Section: Introductionmentioning

confidence: 99%

Generative Autoregressive Networks for 3D Dancing Move Synthesis From Music

Ahn

Kim

Kım

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

This paper proposes a framework which is able to generate a sequence of three-dimensional human dance poses for a given music. The proposed framework consists of three components: a music feature encoder, a pose generator, and a music genre classifier. We focus on integrating these components for generating a realistic 3D human dancing move from music, which can be applied to artificial agents and humanoid robots. The trained dance pose generator, which is a generative autoregressive model, is able to synthesize a dance sequence longer than 5,000 pose frames. Experimental results of generated dance sequences from various songs show how the proposed method generates human-like dancing move to a given music. In addition, a generated 3D dance sequence is applied to a humanoid robot, showing that the proposed framework can make a robot to dance just by listening to music.

show abstract

“…Various kinds of neural network architectures are the main technical basis for the current state of the art for anticipation of human body motions from data [1][2][3][4][5][6][7][8][9]. However, as is the case in many other application domains, there is a fundamental lack of interpretability of the neural networks.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised and Generic Short-Term Anticipation of Human Body Motions

Enes

Errami

Krake

et al. 2020

Sensors

View full text Add to dashboard Cite

Various neural network based methods are capable of anticipating human body motions from data for a short period of time. What these methods lack are the interpretability and explainability of the network and its results. We propose to use Dynamic Mode Decomposition with delays to represent and anticipate human body motions. Exploring the influence of the number of delays on the reconstruction and prediction of various motion classes, we show that the anticipation errors in our results are comparable or even better for very short anticipation times (< 0.4 sec) to a recurrent neural network based method. We perceive our method as a first step towards the interpretability of the results by representing human body motions as linear combinations of "factors". In addition, compared to the neural network based methods large training times are not needed. Actually, our methods do not even regress to any other motions than the one to be anticipated and hence is of a generic nature.

show abstract

Modeling Human Motion with Quaternion-Based Neural Networks

Cited by 149 publications

References 72 publications

Structured Prediction Helps 3D Human Motion Modelling

Structured Prediction Helps 3D Human Motion Modelling

Generative Autoregressive Networks for 3D Dancing Move Synthesis From Music

Unsupervised and Generic Short-Term Anticipation of Human Body Motions

Contact Info

Product

Resources

About