Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitions

Abstract

Human-motion generation is a long-standing challenging task due to the requirement of accurately modeling complex and diverse dynamic patterns. Most existing methods adopt sequence models such as RNN to directly model transitions in the original action space. Due to high dimensionality and potential noise, such modeling of action transitions is particularly challenging. In this paper, we focus on skeleton-based action generation and propose to model smooth and diverse transitions on a latent space of action sequences with much lower dimensionality. Conditioned on a latent sequence, actions are generated by a frame-wise decoder shared by all latent action-poses. Specifically, an implicit RNN is defined to model smooth latent sequences, whose randomness (diversity) is controlled by noise from the input. Different from standard action-prediction methods, our model can generate action sequences from pure noise without any conditional action poses. Remarkably, it can also generate unseen actions from mixed classes during training. Our model is learned with a bi-directional generative-adversarial-net framework, which can not only generate diverse action sequences of a particular class or mix classes, but also learns to classify action sequences within the same model. Experimental results show the superiority of our method in both diverse action-sequence generation and classification, relative to existing methods.

Dynamic Future Net

“…7. Even with the same motion prefix, motions start to diversify from the beginning, which is a distinct property lacking in deterministic generators in most action-prediction models such as [32], and our sequence is in 3d which is more difficult than 2d [43].…”

“…We employ the mean-distance distribution as a measure, as in [43]. For each time step, we calculate the mean pose of all generated motions, then calculate the Euclidean distances between the mean pose and all other poses at that time step.…”

Vision-based 3D human and hand pose analysis

“…and Gaussian Process latent variable models [87] , etc., which commonly suffer from the computational resources and can be easily stuck in non-periodical actions. Recently, due to the success of the sequence-to-sequence inference, RNN-based architectures have been widely used in state-of-the-art approaches [12,[88][89][90][91]. For instance, Fragkiadaki et al [89] proposed a Encoder-Recurrent-Decoder (ERD) framework, which maps pose data into a latent space and propagates it across the temporal domain through LSTM cells.…”

“…To produce long-term 2.4. 3D Pose Datasets robust and dynamic motion synthesis results, generative models [12,13,91,[104][105][106][107][108] have been introduced in this task. Barsoum et al [12] combined the Seq2seq framework with a GAN for motion prediction, which is able to generate multiple results by using different latent vectors drawn from a random distribution.…”

“…On the other hand, generative models have been used in recent approaches [12,13,91,104,105,111] to synthesize plausible motions. However, they are either restricted to one type of motion synthesis or use different approaches to address various tasks of Figure 6.1.…”