Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis

Xu, Tianhan; Fujita, Yasuhiro; Matsumoto, Eiichi

doi:10.1109/cvpr52688.2022.01542

Cited by 21 publications

(12 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The optimization process takes about an hour on a single NVIDIA GTX3090 GPU. The indicative results with plausible quality appear after a few minutes, which is quite faster than our counterparts [70,69,88,92]. Such superior efficiency could largely accelerate downstream applications.…”

Section: Implementation Detailsmentioning

confidence: 73%

“…Baselines. We compare our method with template-based methods [70,92] and template-free methods [69,88]. Here we list the average metric values with different training times to illustrate our very competitive performance and significant speed boost.…”

Section: Evaluation and Comparisonmentioning

confidence: 99%

“…Neural Body (NB) [70] learns a set of latent codes anchored to a deformable template mesh to provide geometry guidance. Surface-Aligned NeRF (SA-NeRF) [92] proposes projecting a point onto a mesh surface to align surface points and signed height to the surface. 2) Template-free methods.…”

Section: Evaluation and Comparisonmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Meshy Neural Fields for Animatable Human Avatars

Huang¹,

Cheng²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Section: Implementation Detailsmentioning

confidence: 73%

Section: Evaluation and Comparisonmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Meshy Neural Fields for Animatable Human Avatars

Huang¹,

Cheng²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…AnimatableNeRF [21] introduces deformation fields based on neural blend weight fields to generate observation-to-canonical correspondences. Surface-Aligned NeRF [35] defines the neural scene representation on the mesh surface points and signed distances from the surface of a human body mesh.…”

Section: Related Workmentioning

confidence: 99%

IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF

Peng¹,

Hu²,

Zhou³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Despite achieving multi-view consistency, most existing 3D-aware generative models are limited to object-centric images, e.g., faces and cars [53,8,7]. There are a few attempts to generate scene images in a compositonal manner [33,44,42,64]. However, all these methods struggle to learn a good geometry of the background and hence do not support large camera movement, e.g., mov-ing the camera along the road.…”

Section: Introductionmentioning

confidence: 99%

UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fields

Yang¹,

Yang²,

Guo³

et al. 2023

Preprint

View full text Add to dashboard Cite

Generating photorealistic images with controllable camera pose and scene contents is essential for many applications including AR/VR and simulation. Despite the fact that rapid progress has been made in 3D-aware generative models, most existing methods focus on object-centric images and are not applicable to generating urban scenes for free camera viewpoint control and scene editing. To address this challenging task, we propose UrbanGIRAFFE, which uses a coarse 3D panoptic prior, including the layout distribution of uncountable stuff and countable objects, to guide a 3D-aware generative model. Our model is compositional and controllable as it breaks down the scene into stuff, objects, and sky. Using stuff prior in the form of semantic voxel grids, we build a conditioned stuff generator that effectively incorporates the coarse semantic and geometry information. The object layout prior further allows us to learn an object generator from cluttered scenes. With proper loss functions, our approach facilitates photorealistic 3D-aware image synthesis with diverse controllability, including large camera movement, stuff editing, and object manipulation. We validate the effectiveness of our model on both synthetic and real-world datasets, including the challenging KITTI-360 dataset. Project page: https:// lv3d.github.io/ urbanGIRAFFE.

show abstract

Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis

Cited by 21 publications

References 22 publications

Efficient Meshy Neural Fields for Animatable Human Avatars

Efficient Meshy Neural Fields for Animatable Human Avatars

IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF

UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fields

Contact Info

Product

Resources

About