Neural Wavelet-domain Diffusion for 3D Shape Generation

Hui, K. C.; Li, Ruihui; Hu, Jingyu; Fu, Chi‐Wing

doi:10.1145/3550469.3555394

Cited by 52 publications

(11 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, neural implicit networks have gained significant attention for their efficacy in 3D representational learning [33,27,1,37,44,42,56,19,17]. While several models have explored implicit representation for 3D surface reconstruction, only a few have used it for 3D model generation [54,17].…”

Section: Implicit Neural Generative Modelsmentioning

confidence: 99%

FullFormer: Generating Shapes Inside Shapes

Medi¹,

Tayyub²,

Sarmad³

et al. 2023

Preprint

View full text Add to dashboard Cite

Implicit generative models have been widely employed to model 3D data and have recently proven to be successful in encoding and generating high-quality 3D shapes. This work builds upon these models and alleviates current limitations by presenting the first implicit generative model that facilitates the generation of complex 3D shapes with rich internal geometric details. To achieve this, our model uses unsigned distance fields to represent nested 3D surfaces allowing learning from non-watertight mesh data. We propose a transformer-based autoregressive model for 3D shape generation that leverages context-rich tokens from vector quantized shape embeddings. The generated tokens are decoded into an unsigned distance field which is rendered into a novel 3D shape exhibiting a rich internal structure. We demonstrate that our model achieves state-of-theart point cloud generation results on popular classes of 'Cars', 'Planes', and 'Chairs' of the ShapeNet dataset. Additionally, we curate a dataset that exclusively comprises shapes with realistic internal details from the 'Cars' class of ShapeNet and demonstrate our method's efficacy in generating these shapes with internal geometry.

show abstract

Section: Implicit Neural Generative Modelsmentioning

confidence: 99%

FullFormer: Generating Shapes Inside Shapes

Medi¹,

Tayyub²,

Sarmad³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Some recent works [HLHF22,ZVW*22,LMW*22] enable shape generation and its counterpart problem shape inversion using latent representations of shapes. However, this approach is applicable in specific use cases such as shape editing, where the procedural model is hidden from the user.…”

Section: Related Workmentioning

confidence: 99%

“…Networks that represent shapes as implicit functions are able to learn models for collections of shapes [PFS*19, CTZ20], but the quality of the results is not optimal; the created shapes are not clean, having unnecessary bumps and dents, while lacking fine details. Recent methods based on transformers or diffusion models provide results that are visually smoother [HPG*22,YLM*22,ZVW*22,HLHF22], but depend on the availability of enough data resembling the target shapes.…”

Section: Introductionmentioning

confidence: 99%

Data‐guided Authoring of Procedural Models of Shapes

Hossain,

Shen,

Igarashi

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Procedural models enable the generation of a large amount of diverse shapes by varying the parameters of the model. However, writing a procedural model for replicating a collection of reference shapes is difficult, requiring much inspection of the original and replicated shapes during the development of the model. In this paper, we introduce a data‐guided method for aiding a programmer in creating a procedural model to replicate a collection of reference shapes. The user starts by writing an initial procedural model, and the system automatically predicts the model parameters for reference shapes, also grouping shapes by how well they are approximated by the current procedural model. The user can then update the procedural model based on the given feedback and iterate the process. Our system thus automates the tedious process of discovering the parameters that replicate reference shapes, allowing the programmer to focus on designing the high‐level rules that generate the shapes. We demonstrate through qualitative examples and a user study that our method is able to speed up the development time for creating procedural models of 2D and 3D man‐made shapes.

show abstract

“…Recent research on 3D generative models has focused primarily on the development of generative models based on VQ-VAE [47,8,73,77,62,72], GAN [4,65,17], or diffusion models [80, 40,50,26]. The present study concentrates on connecting the sketch modality with 3D shapes across three different 3D representations: voxels, CAD, and implicit representation.…”

Section: Related Workmentioning

confidence: 99%