GAN-based reactive motion synthesis with class-aware discriminators for human–human interaction

Men, Qianhui; Shum, Hubert P. H.; Ho, Edmond S. L.; Leung, Howard

doi:10.1016/j.cag.2021.09.014

Cited by 16 publications

(44 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experimental results indicate that the interactions synthesized by our proposed method outperformed the state-of-the-art reaction synthesis model [15] as well as the baselines qualitatively and quantitatively. Numerically, lower Average Frame Distance (AFD) and Fréchet Inception Distance (FID) were obtained using our method which indicates our synthesized motions better resemble the original data.…”

Section: Introductionmentioning

confidence: 89%

Section: Introductionmentioning

confidence: 99%

“…The most relevant research to our work is the GAN-based interaction synthesis framework proposed by Men et al [15]. To the best of our knowledge, [15] is the most recent method that generates the reactive motion of the follower according to the input motion of the leader. [15] considered a seq2seq reactive motion generator and adversarially rectify the synthesized motion with a binary and a multi-class discriminator.…”

Section: Introductionmentioning

confidence: 99%

“…To the best of our knowledge, [15] is the most recent method that generates the reactive motion of the follower according to the input motion of the leader. [15] considered a seq2seq reactive motion generator and adversarially rectify the synthesized motion with a binary and a multi-class discriminator. However, with no label information guided, their model can only produce a fixed reactive pattern learned from the input motion data and fails to create a mixture of reactions.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we introduce a novel concept of using multi-hot class embedding in a conditional GAN to generate higher quality reactive motions with a larger degree of controllability over recent research [15]. The main goal of this work is to enable Interaction Mix and Match which is illustrated in Figure 1.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Interaction Mix and Match: Synthesizing Close Interaction using Conditional Hierarchical GAN with Multi-Hot Class Embedding

Goel¹,

Men²,

Ho³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Synthesizing multi-character interactions is a challenging task due to the complex and varied interactions between the characters. In particular, precise spatiotemporal alignment between characters is required in generating close interactions such as dancing and fighting. Existing work in generating multi-character interactions focuses on generating a single type of reactive motion for a given sequence which results in a lack of variety of the resultant motions. In this paper, we propose a novel way to create realistic human reactive motions which are not presented in the given dataset by mixing and matching different types of close interactions. We propose a Conditional Hierarchical Generative Adversarial Network with Multi-Hot Class Embedding to generate the Mix and Match reactive motions of the follower from a given motion sequence of the leader. Experiments are conducted on both noisy (depth-based) and high-quality (MoCap-based) interaction datasets. The quantitative and qualitative results show that our approach outperforms the state-of-the-art methods on the given datasets. We also provide an augmented dataset with realistic reactive motions to stimulate future research in this area. The code is available at https://github.com/Aman-Goel1/IMM

show abstract

Section: Introductionmentioning

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Interaction Mix and Match: Synthesizing Close Interaction using Conditional Hierarchical GAN with Multi-Hot Class Embedding

Goel¹,

Men²,

Ho³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

A comprehensive review of generative adversarial networks: Fundamentals, applications, and challenges

Mohammed

2023

WIREs Computational Stats

View full text Add to dashboard Cite

In machine learning, a generative model is responsible for generating new samples of data in terms of a probabilistic model. Generative adversarial network (GAN) has been widely used to generate realistic samples in different domains and outperforms its peers in the generative models family. However, producing a robust GAN model is not a trivial task because many challenges face the GAN during the training process and impact its performance, affecting the quality and diversity of the generated samples. In this article, we conduct a comprehensive review of GANs to present the fundamentals of GAN, including its components, types, and objective functions. Also, we present an overview of the evaluation matrices used to evaluate GAN models. Moreover, we list the applications of GANs and research work in various domains. Finally, we present the challenges that face GANs and highlight two significant issues, representing mode collapse and training instability, in addition to those research efforts that tackle these challenges.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Deep Learning Statistical Learning and Exploratory Methods of the Data Sciences > Neural Networks

show abstract

The Role of Generative Artificial Intelligence (GAI) in Education: A Detailed Review for Enhanced Learning Experiences

Kumar,

Kait,

Ankita

et al. 2024

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

GAN-based reactive motion synthesis with class-aware discriminators for human–human interaction

Cited by 16 publications

References 36 publications

Interaction Mix and Match: Synthesizing Close Interaction using Conditional Hierarchical GAN with Multi-Hot Class Embedding

Interaction Mix and Match: Synthesizing Close Interaction using Conditional Hierarchical GAN with Multi-Hot Class Embedding

A comprehensive review of generative adversarial networks: Fundamentals, applications, and challenges

The Role of Generative Artificial Intelligence (GAI) in Education: A Detailed Review for Enhanced Learning Experiences

Contact Info

Product

Resources

About