StylePeople: A Generative Model of Fullbody Human Avatars

Grigorev, Artur; Iskakov, Karim; Ianina, Anastasia; Bashirov, Renat; Zakharkin, Ilya; Vakhitov, Alexander; Lempitsky, Victor

doi:10.1109/cvpr46437.2021.00511

Cited by 57 publications

(43 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1). We show that our learned representation handles occlusions more effectively than other techniques [66,52,54,11] that just take geometry priors (e.g., UV, depth or normal maps) from a coarse mesh as input (see Fig. 3, 7).…”

Section: Related Workmentioning

confidence: 92%

“…A 2D convolutional network is often utilized for both shape completion and appearance synthesis in one stage [66,52]. However, [66,52,54,11] do not reconstruct geometry explicitly and cannot handle self-occlusions effectively. In contrast, our rendering is conditioned on a learned 3D volumetric representation using a two-stage approach (see Fig.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars

Hu¹,

Chen²,

Zheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose a novel neural rendering pipeline, Hybrid Volumetric-Textural Rendering (HVTR), which synthesizes virtual human avatars from arbitrary poses efficiently and at high quality. First, we learn to encode articulated human motions on a dense UV manifold of the human body surface. To handle complicated motions (e.g., self-occlusions), we then leverage the encoded information on the UV manifold to construct a 3D volumetric representation based on a dynamic pose-conditioned neural radiance field. While this allows us to represent 3D geometry with changing topology, volumetric rendering is computationally heavy. Hence we employ only a rough volumetric representation using a pose-conditioned downsampled neural radiance field (PD-NeRF), which we can render efficiently at low resolutions. In addition, we learn 2D textural features that are fused with rendered volumetric features in image space. The key advantage of our approach is that we can then convert the fused features into a high resolution, high quality avatar by a fast GAN-based textural renderer. We demonstrate that hybrid rendering enables HVTR to handle complicated motions, render high quality avatars under usercontrolled poses/shapes and even loose clothing, and most importantly, be fast at inference time. Our experimental results also demonstrate state-of-the-art quantitative results. More results are available at our project page: https: // www. cs. umd. edu/ ~taohu/ hvtr/ .

show abstract

Section: Related Workmentioning

confidence: 92%

Section: Related Workmentioning

confidence: 99%

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars

Hu¹,

Chen²,

Zheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In human generation research, most of the existing applications focus on precise control of pose and appearance by leveraging conditional VAE and U-Net [17,73] or StyleGAN-related architectures [3,24,44,72]. Specifically, the 3D method [24] renders StyleGAN-generated neural textures on the parametric human models, but the results are restricted by the quantity and quality of training data. The other works [3,72,73] preserve texture quality by spatial modulation using the extracted UV texture map, and perform pose transfer conditioned by extracted pose features.…”

Section: Human Generationmentioning

confidence: 99%

StyleGAN-Human: A Data-Centric Odyssey of Human Generation

Fu¹,

Li²,

Jiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…2D and 3D Generative Models: Most modern methods for synthesizing natural images leverage generative adversarial networks (GANs) [15] or variational auto-encoders (VAEs) [24]. These methods have achieved a high level of photorealism [21][22][23] and can yield impressive results on the task of synthesizing 2D images of humans [4,16,26,31,50]. However, such methods reason in 2D and hence 3D consistency cannot be guaranteed [26,31] nor is extracting 3D geometry from such approaches straightforward.…”

Section: Related Workmentioning

confidence: 99%

gDNA: Towards Generative Detailed Neural Avatars

Chen¹,

Jiang²,

Song³

et al. 2022

Preprint

View full text Add to dashboard Cite

https://xuchen-ethz.github.io/gdna Figure 1. Generative Detailed Neural Avatars. We propose a method to generate 1) a diverse set of 3D virtual humans of 2) varied identity, gender and shapes, appearing in 3) different clothing styles and poses, with 4) realistic and stochastic details such as wrinkles in garments. Our multi-subject method learns shape, articulation and clothing details from few posed scans without requiring skinning weight supervision. The method is able to synthesize novel identities that are not in the training set and generalizes to unseen poses.

show abstract

StylePeople: A Generative Model of Fullbody Human Avatars

Cited by 57 publications

References 37 publications

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars

StyleGAN-Human: A Data-Centric Odyssey of Human Generation

gDNA: Towards Generative Detailed Neural Avatars

Contact Info

Product

Resources

About