Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies

Peng, Sida; Dong, Junting; Wang, Qianqian; Zhang, Shangzhan; Shuai, Qing; Zhou, Xiaowei; Bao, Hujun

doi:10.48550/arxiv.2105.02872

Cited by 6 publications

(11 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Particularly, modeling shapes as level sets of neural networks has recently demonstrated to be an effective geometric surface representation [27], [24], [15], [33]. Furthermore, neural implicit models can be used to represent dynamic radiance fields with applications to high-quality rendering and animation based on real world data [28], [31], [30], [29] .…”

Section: Related Workmentioning

confidence: 99%

Neural Implicit Surface Evolution using Differential Equations

Novello¹,

Silva²,

Lopes³

et al. 2022

Preprint

View full text Add to dashboard Cite

This work investigates the use of neural networks admitting high-order derivatives for modeling dynamic variations of smooth implicit surfaces. For this purpose, it extends the representation of differentiable neural implicit surfaces to higher dimensions, which opens up mechanisms that allow to exploit geometric transformations in many settings, from animation and surface evolution to shape morphing and design galleries.The problem is modeled by a k-parameter family of surfaces Sc, specified as a neural network function f : R 3 × R k → R, where Sc is the zero-level set of the implicit function f (•, c) : R 3 → R, with c ∈ R k , with variations induced by the control variable c. In that context, restricted to each coordinate of R k , the underlying representation is a neural homotopy which is the solution of a general partial differential equation.

show abstract

Section: Related Workmentioning

confidence: 99%

Neural Implicit Surface Evolution using Differential Equations

Novello¹,

Silva²,

Lopes³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Aiming at human body, several methods have been proposed by introducing human parametric model (e.g. SMPL) [6,28,32,36] or skeleton [35] as prior to build NeRF for human body. For a wide range of dynamic scenarios, Park et al [33] proposed to augment NeRF by optimizing an additional continuous volumetric deformation field, while Pumarola et al [37] optimized an underlying deformable volumetric function.…”

Section: Related Workmentioning

confidence: 99%

MoFaNeRF: Morphable Facial Neural Radiance Field

Zhuang¹,

Zhu²,

Sun³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent work has also focused on modeling dynamic scenes, mostly with monocular videos as input [25,15,27,4,9,34]. Most approaches [25,27,4,26] combine a canonical model of the object with a deformation network, or warp space [34], still starting from a canonical volume. Closer to our work, [15] and [9] combine a static NeRF model and a dynamic one.…”

Section: Related Workmentioning

confidence: 99%

NeuralDiff: Segmenting 3D objects that move in egocentric videos

Tschernezki¹,

Larlus²,

Vedaldi³

2021

Preprint

View full text Add to dashboard Cite

Figure 1: Given a video sequence captured from an egocentric viewpoint, we segment all the objects that the actor/observer interacts with. We achieve this by means of a neural architecture, NeuralDiff, that learns to decompose each frame into a static background and a dynamic foreground, comprising the manipulated objects, which seldomly move in the sequence, and the actor's body, which moves continually and heavily occludes the scene. The neural network contains three streams that, via different inductive biases, reconstruct the background, the objects and the actor in 3D, and is thus able to render images and their segmentations also for viewpoints that do not exist in the original video sequence (see the left part of the figure, where the camera is assumed to remain at its initial position while the action unfolds).

show abstract

Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies

Cited by 6 publications

References 57 publications

Neural Implicit Surface Evolution using Differential Equations

Neural Implicit Surface Evolution using Differential Equations

MoFaNeRF: Morphable Facial Neural Radiance Field

NeuralDiff: Segmenting 3D objects that move in egocentric videos

Contact Info

Product

Resources

About