NPMs: Neural Parametric Models for 3D Deformable Shapes

Palafox, Pablo; Božič, Aljaž; Thies, Justus; Nießner, Matthias; Dai, Angela

doi:10.48550/arxiv.2104.00702

Cited by 10 publications

(10 citation statements)

References 35 publications

(82 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the complexity of clothes and accessories, non-parametric human modelings [Alldieck et al 2018;Burov et al 2021;Corona et al 2021;Huang et al 2020;Mihajlovic et al 2021;Palafox et al 2021;Saito et al 2021;Weng et al 2019] are proposed to offer more flexibility on realistic human modeling. Moreover, inspired by recent advances in volume rendering, nonparametric human modeling methods based on the neural radiance field (i.e.…”

Section: Related Workmentioning

confidence: 99%

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

Hong¹,

Zhang²,

Peng³

et al. 2022

Preprint

View full text Add to dashboard Cite

texture generation. Moreover, by leveraging the priors learned in the motion VAE, a CLIP-guided reference-based motion synthesis method is proposed for the animation of the generated 3D avatar. Extensive qualitative and quantitative experiments validate the effectiveness and generalizability of AvatarCLIP on a wide range of avatars. Remarkably, AvatarCLIP can generate unseen 3D avatars with novel animations, achieving superior zero-shot capability. Codes are available at https://github.com/hongfz16/AvatarCLIP.

show abstract

Section: Related Workmentioning

confidence: 99%

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

Hong¹,

Zhang²,

Peng³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…There are two potential solutions to shorten the inference time: 1) as [39], apply modulation network maps a latent code to the modification of parameters of a base network. ; 2) as NPMs [44], train an encoder separately to overfit the learned latent code and utilize this encoder to generate the initialization for latent code optimization.…”

Section: Limitationsmentioning

confidence: 99%

Topology-Preserving Shape Reconstruction and Registration via Neural Diffeomorphic Flow

Sun¹,

Han²,

Kong³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep Implicit Functions (DIFs) represent 3D geometry with continuous signed distance functions learned through deep neural nets. Recently DIFs-based methods have been proposed to handle shape reconstruction and dense point correspondences simultaneously, capturing semantic relationships across shapes of the same class by learning a DIFs-modeled shape template. These methods provide great flexibility and accuracy in reconstructing 3D shapes and inferring correspondences. However, the point correspondences built from these methods do not intrinsically preserve the topology of the shapes, unlike meshbased template matching methods. This limits their applications on 3D geometries where underlying topological structures exist and matter, such as anatomical structures in medical images. In this paper, we propose a new model called Neural Diffeomorphic Flow (NDF) to learn deep implicit shape templates, representing shapes as conditional diffeomorphic deformations of templates, intrinsically preserving shape topologies. The diffeomorphic deformation is realized by an auto-decoder consisting of Neural Ordinary Differential Equation (NODE) blocks that progressively map shapes to implicit templates. We conduct extensive experiments on several medical image organ segmentation datasets to evaluate the effectiveness of NDF on reconstructing and aligning shapes. NDF achieves consistently state-of-the-art organ shape reconstruction and registration results in both accuracy and quality. The source code is publicly available at https : / / github . com / Siwensun / Neural _ Diffeomorphic_Flow--NDF.

show abstract

“…However, such parametric models requires substantial efforts from experts to construct and thus is hard to generalize to large-scale object categories. To address the challenge, another line of work [59,55,77,12,8,57] employs neural networks to learn shapes from data. For example, Niemeyer et al [55] learned an implicit vector field assigning every point with a motion vector and deformed shapes in a spatialtemporal space.…”

Section: Related Workmentioning

confidence: 99%

A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation

Kortylewski

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent work has made significant progress on using implicit functions, as a continuous representation for 3D rigid object shape reconstruction. However, much less effort has been devoted to modeling general articulated objects. Compared to rigid objects, articulated objects have higher degrees of freedom, which makes it hard to generalize to unseen shapes. To deal with the large shape variance, we introduce Articulated Signed Distance Functions (A-SDF) to represent articulated shapes with a disentangled latent space, where we have separate codes for encoding shape and articulation. We assume no prior knowledge on part geometry, articulation status, joint type, joint axis, and joint location. With this disentangled continuous representation, we demonstrate that we can control the articulation input and animate unseen instances with unseen joint angles. Furthermore, we propose a Test-Time Adaptation inference algorithm to adjust our model during inference. We demonstrate our model generalize well to out-of-distribution and unseen data, e.g., partial point clouds and real-world depth images. Project page: https://jitengmu.github.io/A-SDF/.

show abstract

NPMs: Neural Parametric Models for 3D Deformable Shapes

Cited by 10 publications

References 35 publications

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

Topology-Preserving Shape Reconstruction and Registration via Neural Diffeomorphic Flow

A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation

Contact Info

Product

Resources

About