LIMP: Learning Latent Shape Representations with Metric Preservation Priors

Cosmo, Luca; Norelli, Antonio; Halimi, Oshri; Kimmel, Ron; Rodolà, Emanuele

doi:10.1007/978-3-030-58580-8_2

Cited by 47 publications

(52 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Towards 3D mesh deformation, previous works demand re-enforcing the correspondence between source and target meshes. For example, some disentanglement-based methods like [10,36] use the shape correspondences between different pose meshes from the same body to decompose shape and pose factors. SNARF [7] uses differentiable forward skinning for animating nonrigid neural implicit shapes, which can be extended to unseen and complex shapes, but bone transformations between the target meshes and the desired pose are critical in their settings.…”

Section: Related Workmentioning

confidence: 99%

“…In Eq. ( 6), reconstruction loss L r is a simple point-wise L2 distance commonly used in previous works [10,31,36].…”

Section: Imposing Regression Constraintsmentioning

confidence: 99%

“…We compare the proposed approach to the most recent single mesh deformation approaches. Specifically, we compare with two state-of-the-art methods, i.e., NPT [31] and LIMP [10]. Since both methods could only process a single mesh, we use them to generate long sequences frame by frame.…”

Section: Datasetsmentioning

confidence: 99%

“…Using a learned 3D animating model in virtual reality (VR) scenarios, one can generate realistic animated 3D avatars without explicitly specifying the geometry properties and dynamics, which would be cumbersome but inevitable when using a standard graphics rendering engine. There are various existing works related to 3D animation problem, including 3D mesh deformation [1,7,10,12,24,26,31,36], 3D motion estimation [9,17,18], and 3D motion synthesis [21,22]. In this paper, we study a new form, i.e., data-driven 3D animation.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

AniFormer: Data-driven 3D Animation with Transformer

Chen¹,

Tang²,

Sebe³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a novel task, i.e., animating a target 3D object through the motion of a raw driving sequence. In previous works, extra auxiliary correlations between source and target meshes or intermedia factors are inevitable to capture the motions in the driving sequences. Instead, we introduce AniFormer, a novel Transformer-based architecture, that generates animated 3D sequences by directly taking the raw driving sequences and arbitrary same-type target meshes as inputs. Specifically, we customize the Transformer architecture for 3D animation that generates mesh sequences by integrating styles from target meshes and motions from the driving meshes. Besides, instead of the conventional single regression head in the vanilla Transformer, AniFormer generates multiple frames as outputs to preserve the sequential consistency of the generated meshes. To achieve this, we carefully design a pair of regression constraints, i.e., motion and appearance constraints, that can provide strong regularization on the generated mesh sequences. Our AniFormer achieves high-fidelity, realistic, temporally coherent animated results and outperforms compared start-of-the-art methods on benchmarks of diverse categories. Code is available: https://github.com/mikecheninoulu/AniFormer.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In Eq. ( 6), reconstruction loss L r is a simple point-wise L2 distance commonly used in previous works [10,31,36].…”

Section: Imposing Regression Constraintsmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

AniFormer: Data-driven 3D Animation with Transformer

Chen¹,

Tang²,

Sebe³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, to provide those correspondences needs either extra manual efforts or specific requirements for data. Although some existing works claimed that they can achieve 3D pose deformation in an unsupervised setting, a constraint on the training datasets is still needed that different poses performed by the same subject should be given to successfully disentangle the shape and pose information [4,39]. This constraint actually serves as a strong supervision with manually labeled prior.…”

Section: Introductionmentioning

confidence: 99%

Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer

Chen

Tang

Shi

et al. 2021

Preprint

View full text Add to dashboard Cite

With the strength of deep generative models, 3D pose transfer regains intensive research interests in recent years. Existing methods mainly rely on a variety of constraints to achieve the pose transfer over 3D meshes, e.g., the need for manually encoding for shape and pose disentanglement. In this paper, we present an unsupervised approach to conduct the pose transfer between any arbitrate given 3D meshes. Specifically, a novel Intrinsic-Extrinsic Preserved Generative Adversarial Network (IEP-GAN) is presented for both intrinsic (i.e., shape) and extrinsic (i.e., pose) information preservation. Extrinsically, we propose a cooccurrence discriminator to capture the structural/pose invariance from distinct Laplacians of the mesh. Meanwhile, intrinsically, a local intrinsic-preserved loss is introduced to preserve the geodesic priors while avoiding heavy computations. At last, we show the possibility of using IEP-GAN to manipulate 3D human meshes in various ways, including pose transfer, identity swapping and pose interpolation with latent code vector arithmetic. The extensive experiments on various 3D datasets of humans, animals and hands qualitatively and quantitatively demonstrate the generality of our approach. Our proposed model produces better results and is substantially more efficient compared to recent state-ofthe-art methods. Code is available: https://github. com/mikecheninoulu/Unsupervised_IEPGAN

show abstract

3D mesh pose transfer based on skeletal deformation

Yang

Yin

et al. 2023

Computer Animation & Virtual

View full text Add to dashboard Cite

For 3D mesh pose transfer, the target model is obtained by transferring the pose of the reference mesh to the source mesh, where the shape and pose of the source are usually different from that of the reference. In this paper, pose transfer is considered as a deformation process of the source mesh, and we propose a 3D mesh pose transfer method based on skeletal deformation. First, we design a neural network based on the edge convolution operator to extract the skeleton of the 3D mesh and bind the rigid weights; then, we calculate the bone transformations between the two skeletons with different poses and use the diffusion equation to smooth the rigid weights; finally, the source mesh is deformed according to the bone transformations and the smooth weights to get the target mesh. Experiment results on different datasets show that the pose of the reference mesh can be effectively transferred to the source one while maintaining the shape and high‐quality geometric details of the source mesh by using our method.

show abstract

LIMP: Learning Latent Shape Representations with Metric Preservation Priors

Cited by 47 publications

References 39 publications

AniFormer: Data-driven 3D Animation with Transformer

AniFormer: Data-driven 3D Animation with Transformer

Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer

3D mesh pose transfer based on skeletal deformation

Contact Info

Product

Resources

About