Multi-chart generative surface modeling

Abstract

Figure 1: Our method is able to learn shape distribution and generate unseen shapes. This figure shows 1024 human models randomly generated by our method. AbstractThis paper introduces a 3D shape generative model based on deep neural networks. A new image-like (i.e., tensor) data representation for genus-zero 3D shapes is devised. It is based on the observation that complicated shapes can be well represented by multiple parameterizations (charts), each focusing on a different part of the shape. The new tensor data representation is used as input to Generative Adversarial Networks for the task of 3D shape generation.The 3D shape tensor representation is based on a multichart structure that enjoys a shape covering property and scale-translation rigidity. Scale-translation rigidity facilitates high quality 3D shape learning and guarantees unique reconstruction. The multi-chart structure uses as input a dataset of 3D shapes (with arbitrary connectivity) and a sparse correspondence between them. The output of our algorithm is a generative model that learns the shape distribution and is able to generate novel shapes, interpolate shapes, and explore the generated shape space. The effectiveness of the method is demonstrated for the task of anatomic shape generation including human body and bone (teeth) shape generation.

A survey on deep geometry learning: From a representation perspective

“…AtlasNet learns to convert 2D square patches into 2-manifolds to cover the surface of 3D shapes using an MLP (multi-layer perceptron). Ben-Hamu et al [87] proposed a multichart generative model for 3D shape generation. It uses a multi-chart structure as input; the network architecture is based on standard image GAN [41].…”

OperatorNet: Recovering 3D Shapes From Difference Operators

“…In contrast, interpolation using OperatorNet is continuous and respects the structure and constraints of the body, suggesting that shape differences efficiently encode the shape structure. We provide further comparisons to other baselines including [30,3,12] and to linear interpolation of shape differences in Appendix E.…”

“…On the other hand, in [3], a GAN is trained to generate realistic human shapes. In particular, we follow the interpolation scheme described in [3]: first we pick two randomly generated latent vectors z 1 , z 2 , which, via the GAN give arise to two shapes G(z 1 ), G(z 2 ). Then, the interpolation between the two shapes is achieved as G(z(t)), where z(t) = (1 − t)z 1 + tz 2 .…”

“…This method generates significant distortions during the interpolation, particularly on the arms. In the second row, note that the interpolations from Multi-Chart GAN [3] between the two end shapes are not evenly spaced. For instance, the arms change abruptly during the three middle shapes, while there is little change on that region afterwards.…”

TileGAN

“…While traditional GANs generate images starting from a random vector, the GAN training can be extended to the problem of image-to-image translation using either paired or unpaired training data Isola et al 2017;Zhu et al 2017a,b]. In computer graphics, recent papers apply GANs to the synthesis of caricatures of human faces [Cao et al 2018], the synthesis of human avatars from a single image [Nagano et al 2018], texture and geometry synthesis of building details [Kelly et al 2018], surface-based modeling of shapes [Ben-Hamu et al 2018] and the volumetric modeling of shapes [Wang et al 2018a]. The most related problem to our work is the problem of terrain synthesis [Guérin et al 2017].…”