BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN

Abstract

Human motion prediction model has applications in various fields of computer vision. Without taking into account the inherent stochasticity in the prediction of future pose dynamics, such methods often converges to a deterministic undesired mean of multiple probable outcomes. Devoid of this, we propose a novel probabilistic generative approach called Bidirectional Human motion prediction GAN, or BiHMP-GAN. To be able to generate multiple probable human-pose sequences, conditioned on a given starting sequence, we introduce a random extrinsic factor r, drawn from a predefined prior distribution. Furthermore, to enforce a direct content loss on the predicted motion sequence and also to avoid mode-collapse, a novel bidirectional framework is incorporated by modifying the usual discriminator architecture. The discriminator is trained also to regress this extrinsic factor r, which is used alongside with the intrinsic factor (encoded starting pose sequence) to generate a particular pose sequence. To further regularize the training, we introduce a novel recursive prediction strategy. In spite of being in a probabilistic framework, the enhanced discriminator architecture allows predictions of an intermediate part of pose sequence to be used as a conditioning for prediction of the latter part of the sequence. The bidirectional setup also provides a new direction to evaluate the prediction quality against a given test sequence. For a fair assessment of BiHMP-GAN, we report performance of the generated motion sequence using (i) a critic model trained to discriminate between real and fake motion sequence, and (ii) an action classifier trained on real human motion dynamics. Outcomes of both qualitative and quantitative evaluations, on the probabilistic generations of the model, demonstrate the superiority of BiHMP-GAN over previously available methods.

An Information-Theoretic Explanation for the Adversarial Fragility of AI Classifiers

“…While we present some simple experimental results to validate our theory, our focus here is on the theoretical ideas; the important and challenging problem of designing good generative models to implement the proposed defense for general AI classification systems is deferred to future work. Interestingly, while our theory is novel, other researchers have recently developed defenses for AI classifiers against adversarial attacks that are consistent with our proposed approach [11], [12]. An AI classifier can be defined as a system that takes a high-dimensional vector as input and maps it to a discrete set of labels.…”

A Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction

“…In heuristic research for motion prediction, a representative residual network [8] was first proposed to estimate velocity, which has achieved great success in reducing initial discontinuity of the generated sequence compared with previous attempts [30], [31] predicting only static poses. This triggers many sequential-based motion prediction frameworks [13], [16], [32] introducing residual connection into their baselines. One step of residual connection means that the system outputs velocity from the pose, and adds the velocity back to the previous pose to predict the next step.…”

“…Comparing with the state of the art, we test on Mean Angle Error (MAE) as previous motion prediction works [8], [15], [16]. We also verify the predicted sequence with positionbased metrics, i.e.…”

“…When synthesizing future movements, the discontinuity problem describes the irregular jump between the given motion and prediction. Residual connections [8], [15], [16] attempted to eliminate this by modeling the static poses to predict the dynamic velocities, where the discontinuity still exists as the motion dynamics is indeed not observed by the model. This motivates us to train the velocity in a consistent way, i.e.…”

Densely connected GCN model for motion prediction

“…There were a lot of researchers attempting to propose various approaches for motion prediction. Most deep learning models treat motion prediction similar to machine translation problems and employ long short‐term memory (LSTM)‐ or convolutional neural network (CNN)‐based models 9‐16 . However, different from machine translation, motion data has special human body constraints and is actually a spatial‐temporal data rather than temporal data.…”