A Survey on Deep Learning for Skeleton‐Based Human Animation

Mourot, Lucas; Hoyet, Ludovic; Clerc, François; Schnitzler, François; Hellier, Pierre

doi:10.1111/cgf.14426

Cited by 42 publications

(42 citation statements)

References 150 publications

(499 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We review various relevant works on motion generation, focusing on human and other character animation. For an in-depth survey of this extensive body of literature we refer the readers to the surveys in [Geijtenbeek et al 2011;Mourot et al 2021].…”

Section: Related Workmentioning

confidence: 99%

GANimator: Neural Motion Synthesis from a Single Sequence

Li,

Aberman,

Zhang

et al. 2022

Preprint

View full text Add to dashboard Cite

Fig. 1. Given a single training motion sequence of skeleton with arbitrary topology, our method learns to synthesize novel motion sequences.We present GANimator, a generative model that learns to synthesize novel motions from a single, short motion sequence. GANimator generates motions that resemble the core elements of the original motion, while simultaneously synthesizing novel and diverse movements. Existing data-driven techniques for motion synthesis require a large motion dataset which contains the desired and specific skeletal structure. By contrast, GANimator only requires training on a single motion sequence, enabling novel motion synthesis for a variety of skeletal structures e.g., bipeds, quadropeds, hexapeds, and more. Our framework contains a series of generative and adversarial neural networks, each responsible for generating motions in a specific frame rate. The framework progressively learns to synthesize motion from random noise, enabling hierarchical control over the generated motion content across varying levels of detail. We show a number of applications, including crowd simulation, key-frame editing, style transfer, and interactive control, which all learn from a single input sequence. Code and data for this paper are at https://peizhuoli.github.io/ganimator. CCS Concepts: • Computing methodologies → Motion processing; Machine learning approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

GANimator: Neural Motion Synthesis from a Single Sequence

Li,

Aberman,

Zhang

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, a challenge in using recurrent models is that errors in the predicted poses are fed back into the input and accumulates [41], [45], causing them to fail in long-term motion prediction. A commonly observed consequence of this is that despite an RNN dominating in short-term prediction [14], the motions eventually collapse to an average pose [14], [45], [58].…”

Section: B Recurrent Modelsmentioning

confidence: 99%

“…Phase functions may not generalise well to other less repetitive motion types as they require the segmentation of each motion by start and endpoints, which in practice are difficult to define [58]. In this case, rather than letting the model map the differences between each motion internally, an external phase function needs to be manually defined for each motion [45].…”

Section: Phase Functioned Neural Networkmentioning

confidence: 99%

“…This approach to generator training is powerful because rather than learning to recreate the specific data samples in a given dataset, the generative network learns to generate data that could plausibly have been sampled from the true data population. Although any motion generation network can be wrapped into a GAN by incorporating a discriminator into the training [45], a popular architecture for GANs is based on adversarial autoencoders [38]. The motion is first encoded into a latent code, from which future motions are predicted using the decoder.…”

Section: F Generative Adversarial Networkmentioning

confidence: 99%

See 1 more Smart Citation

Adversarial Attention for Human Motion Synthesis

Malek-Podjaski¹,

Deligianni²

2022

Preprint

View full text Add to dashboard Cite

Analysing human motions is a core topic of interest for many disciplines, from Human-Computer Interaction, to entertainment, Virtual Reality and healthcare. Deep learning has achieved impressive results in capturing human pose in realtime. On the other hand, due to high inter-subject variability, human motion analysis models often suffer from not being able to generalise to data from unseen subjects due to very limited specialised datasets available in fields such as healthcare. However, acquiring human motion datasets is highly timeconsuming, challenging, and expensive. Hence, human motion synthesis is a crucial research problem within deep learning and computer vision. We present a novel method for controllable human motion synthesis by applying attention-based probabilistic deep adversarial models with end-to-end training. We show that we can generate synthetic human motion over both short-and long-time horizons through the use of adversarial attention. Furthermore, we show that we can improve the classification performance of deep learning models in cases where there is inadequate real data, by supplementing existing datasets with synthetic motions.

show abstract

“…In the past years, research on non-verbal social behavior forecasting has followed distinct paths for social signal prediction and computer vision fields, although they share most of their fundamental concerns. For example, the human motion forecasting field does not usually refer to any social signal forecasting work (Mourot et al, 2021), even though some of them predict visual social cues, or action labels. And vice versa.…”

Section: Introductionmentioning

confidence: 99%

Didn't see that coming: a survey on non-verbal social human behavior forecasting

Barquero¹,

Núñez²,

Escalera³

et al. 2022

Preprint

View full text Add to dashboard Cite

Non-verbal social human behavior forecasting has increasingly attracted the interest of the research community in recent years. Its direct applications to human-robot interaction and socially-aware human motion generation make it a very attractive field. In this survey, we define the behavior forecasting problem for multiple interactive agents in a generic way that aims at unifying the fields of social signals prediction and human motion forecasting, traditionally separated. We hold that both problem formulations refer to the same conceptual problem, and identify many shared fundamental challenges: future stochasticity, context awareness, history exploitation, etc. We also propose a taxonomy that comprises methods published in the last 5 years in a very informative way and describes the current main concerns of the community with regard to this problem. In order to promote further research on this field, we also provide a summarized and friendly overview of audiovisual datasets featuring non-acted social interactions. Finally, we describe the most common metrics used in this task and their particular issues.

show abstract

A Survey on Deep Learning for Skeleton‐Based Human Animation

Cited by 42 publications

References 150 publications

GANimator: Neural Motion Synthesis from a Single Sequence

GANimator: Neural Motion Synthesis from a Single Sequence

Adversarial Attention for Human Motion Synthesis

Didn't see that coming: a survey on non-verbal social human behavior forecasting

Contact Info

Product

Resources

About