Learning Gradient Fields for Shape Generation

Cai, Ruojin; Yang, Guandao; Averbuch‐Elor, Hadar; Hao, Zekun; Belongie, Serge; Snavely, Noah; Hariharan, Bharath

doi:10.48550/arxiv.2008.06520

Cited by 7 publications

(14 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PointGrow [21] is an auto-regressive model with exact likelihoods. More recently, [2] proposed a score-matching energy-based model ShapeGF to model the distribution of points.…”

Section: Related Workmentioning

confidence: 99%

“…We quantitatively compare our method with the following state-of-the-art generative models: PC-GAN [1], GCN-GAN [22], TreeGAN [19], PointFlow [25] and ShapeGF [2] using point clouds from two categories in ShapeNet: airplane and chair. Following ShapeGF [2], when evaluating each of the model, we normalize both generated point clouds and reference point clouds into a bounding box of [−1, 1] 3 , so that the metrics focus on the shape of point clouds but not the scale. We evaluate the point clouds generated by the models using the metrics in Section 5.1 and summarize the results in Table 1.…”

Section: Point Cloud Generationmentioning

confidence: 99%

“…We evaluate the reconstruction quality of the proposed auto-encoder, with comparisons against state-of-theart point cloud auto-encoders: AtlasNet [10], PointFlow [25] and ShapeGF [2]. Four datasets are used in the evaluation, which include three categories in ShapeNet: air- plane, car, chair and the whole ShapeNet.…”

Section: Point Cloud Auto-encodingmentioning

confidence: 99%

See 2 more Smart Citations

Diffusion Probabilistic Models for 3D Point Cloud Generation

Luo¹,

Hu²

2021

Preprint

View full text Add to dashboard Cite

We present a probabilistic model for point cloud generation, which is critical for various 3D vision tasks such as shape completion, upsampling, synthesis and data augmentation. Inspired by the diffusion process in non-equilibrium thermodynamics, we view points in point clouds as particles in a thermodynamic system in contact with a heat bath, which diffuse from the original distribution to a noise distribution. Point cloud generation thus amounts to learning the reverse diffusion process that transforms the noise distribution to the distribution of a desired shape. Specifically, we propose to model the reverse diffusion process for point clouds as a Markov chain conditioned on certain shape latent. We derive the variational bound in closed form for training and provide implementations of the model. Experimental results demonstrate that our model achieves the state-of-the-art performance in point cloud generation and auto-encoding. The code is available at https://github.com/luost26/diffusionpoint-cloud

show abstract

“…PointGrow [21] is an auto-regressive model with exact likelihoods. More recently, [2] proposed a score-matching energy-based model ShapeGF to model the distribution of points.…”

Section: Related Workmentioning

confidence: 99%

Section: Point Cloud Generationmentioning

confidence: 99%

See 1 more Smart Citation

Diffusion Probabilistic Models for 3D Point Cloud Generation

Luo¹,

Hu²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In another work [30], the authors propose to model a shape by learning the gradient field of its log-density. [26] uses auto-regressive architecture to model the distribution of 3D points, while in [31] the authors leverage a GAN architecture for the same purpose.…”

Section: Generating 3d Objectsmentioning

confidence: 99%

HyperColor: A HyperNetwork Approach for Synthesizing Auto-colored 3D Models for Game Scenes Population

Kostiuk¹,

Stachura²,

Tadeja³

et al. 2021

Preprint

View full text Add to dashboard Cite

Designing a 3D game scene is a tedious task that often requires a substantial amount of work. Typically, this task involves synthesis, coloring, and placement of 3D models within the game scene. To lessen this workload, we can apply machine learning to automate some aspects of the game scene development. Earlier research has already tackled automated generation of the game scene background with machine learning. However, model auto-coloring remains an underexplored problem. The automatic coloring of a 3D model is a challenging task, especially when dealing with the digital representation of a colorful, multipart object. In such a case, we have to "understand" the object's composition and coloring scheme of each part. Existing single-stage methods have their own caveats such as the need for segmentation of the object or generating individual parts that have to be assembled together to yield the final model. We address these limitations by proposing a two-stage training approach to synthesize auto-colored 3D models. In the first stage, we obtain a 3D point cloud representing a 3D object, whilst in the second stage, we assign colors to points within such cloud. Next, by leveraging the so-called triangulation trick, we generate a 3D mesh in which the surfaces are colored based on interpolation of colored points representing vertices of a given mesh triangle. This approach allows us to generate a smooth coloring scheme. Experimental evaluation shows that our two-stage approach gives better results in terms of shape reconstruction and coloring when compared to traditional single-stage techniques.

show abstract

“…These models have achieved ImageNet generation results outperforming BigGAN-deep and VQ-VAE-2 in terms of FID score and classification accuracy score Dhariwal & Nichol, 2021), and they have achieved likelihoods outperforming autoregressive image models (Kingma et al, 2021;Song et al, 2021b). Diffusion models have also succeeded in image super-resolution and image inpainting (Song et al, 2021c), and there have been promising results in shape generation (Cai et al, 2020), graph generation (Niu et al, 2020), and text generation (Hoogeboom et al, 2021;Austin et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Progressive Distillation for Fast Sampling of Diffusion Models

Salimans¹,

Ho²

2022

Preprint

View full text Add to dashboard Cite

Diffusion models have recently shown great promise for generative modeling, outperforming GANs on perceptual quality and autoregressive models at density estimation. A remaining downside is their slow sampling time: generating high quality samples takes many hundreds or thousands of model evaluations. Here we make two contributions to help eliminate this downside: First, we present new parameterizations of diffusion models that provide increased stability when using few sampling steps. Second, we present a method to distill a trained deterministic diffusion sampler, using many steps, into a new diffusion model that takes half as many sampling steps. We then keep progressively applying this distillation procedure to our model, halving the number of required sampling steps each time. On standard image generation benchmarks like CIFAR-10, ImageNet, and LSUN, we start out with state-of-the-art samplers taking as many as 8192 steps, and are able to distill down to models taking as few as 4 steps without losing much perceptual quality; achieving, for example, a FID of 3.0 on CIFAR-10 in 4 steps. Finally, we show that the full progressive distillation procedure does not take more time than it takes to train the original model, thus representing an efficient solution for generative modeling using diffusion at both train and test time.

show abstract

Learning Gradient Fields for Shape Generation

Cited by 7 publications

References 49 publications

Diffusion Probabilistic Models for 3D Point Cloud Generation

Diffusion Probabilistic Models for 3D Point Cloud Generation

HyperColor: A HyperNetwork Approach for Synthesizing Auto-colored 3D Models for Game Scenes Population

Progressive Distillation for Fast Sampling of Diffusion Models

Contact Info

Product

Resources

About