Learning a Neural 3D Texture Space From 2D Exemplars

Henzler, Philipp; Mitra, Niloy J.; Ritschel, Tobias

doi:10.1109/cvpr42600.2020.00838

Cited by 84 publications

(74 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3D CNNs have been proposed to generate voxel colours [40] as well as geometry [51,50], however such have high memory requirements and there are sparse training 3D object datasets with realistic textures. Attempts to overcome these limitations are ongoing, using a variety of approaches [28,21,23,35,43,22]; results are generally low resolution or only in screen-space. GANs have been previously proposed to align and optimize a set of images for a scanned object [24] producing high quality results; however, they require images of a specific object.…”

Section: Related Workmentioning

confidence: 99%

Projective Urban Texturing

Yiangos¹,

Melinos²,

Kelly³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Projective Urban Texturing

Yiangos¹,

Melinos²,

Kelly³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…These methods can search for suitable textures or materials in a database but are not designed to generate novel textures. Henzler et al [2020] propose a method to encode stochastic textures from 2D exemplars for synthesizing 3D solid textures. While solid textures are useful for certain applications, they are not associated with shape surfaces.…”

Section: Related Workmentioning

confidence: 99%

“…Expansions to other types of input, such as line drawings for both shape and texture, and to other encoding schemes are also interesting to explore. We also plan to extend our current representation with procedural texture generation, e.g., based on Perlin noise style texture [Henzler et al 2020] to improve texture generation. Finally, our current shape-conditioned texturing is unable to generate stylish outputs such as the ones shown in Figure 14 which exhibit significant deviations between textures in different parts.…”

Section: Conclusion Limitation and Future Workmentioning

confidence: 99%

Tm-Net

Gao

Yuan

et al. 2021

ACM Trans. Graph.

View full text Add to dashboard Cite

We introduce TM-NET, a novel deep generative model for synthesizing textured meshes in a part-aware manner. Once trained, the network can generate novel textured meshes from scratch or predict textures for a given 3D mesh, without image guidance. Plausible and diverse textures can be generated for the same mesh part, while texture compatibility between parts in the same shape is achieved via conditional generation. Specifically, our method produces texture maps for individual shape parts, each as a deformable box, leading to a natural UV map with limited distortion. The network separately embeds part geometry (via a PartVAE) and part texture (via a TextureVAE) into their respective latent spaces, so as to facilitate learning texture probability distributions conditioned on geometry. We introduce a conditional autoregressive model for texture generation, which can be conditioned on both part geometry and textures already generated for other parts to achieve texture compatibility. To produce high-frequency texture details, our Tex-tureVAE operates in a high-dimensional latent space via dictionary-based vector quantization. We also exploit transparencies in the texture as an effective means to model complex shape structures including topological details. Extensive experiments demonstrate the plausibility, quality, and diversity of the textures and geometries generated by our network, while avoiding inconsistency issues that are common to novel view synthesis methods. CCS Concepts: • Computing methodologies → Shape modeling.

show abstract

“…• We design the split MLP architecture that achieves faster training and inference and retains the same level of accuracy as the baseline models with comparable or slightly larger parameter counts. (Peng et al, 2020;Mescheder et al, 2019), signed distance (SDF) (Park et al, 2019;Xu et al, 2019), texture (Saito et al, 2019;Henzler et al, 2020), and radiance fields (NeRF) . Unlike MLPs used for other tasks, an MLP used for implicit representation needs to be able to capture the high frequency variations in the color or geometry of images and objects.…”

Section: Introductionmentioning

confidence: 99%

CoordX: Accelerating Implicit Neural Representation with a Split MLP Architecture

Liang¹,

Sun²,

Vijaykumar³

2022

Preprint

View full text Add to dashboard Cite

Implicit neural representations with multi-layer perceptrons (MLPs) have recently gained prominence for a wide variety of tasks such as novel view synthesis and 3D object representation and rendering. However, a significant challenge with these representations is that both training and inference with an MLP over a large number of input coordinates to learn and represent an image, video, or 3D object, require large amounts of computation and incur long processing times. In this work, we aim to accelerate inference and training of coordinate-based MLPs for implicit neural representations by proposing a new split MLP architecture, Co-ordX. With CoordX, the initial layers are split to learn each dimension of the input coordinates separately. The intermediate features are then fused by the last layers to generate the learned signal at the corresponding coordinate point. This significantly reduces the amount of computation required and leads to large speedups in training and inference, while achieving similar accuracy as the baseline MLP. This approach thus aims at first learning functions that are a decomposition of the original signal and then fusing them to generate the learned signal. Our proposed architecture can be generally used for many implicit neural representation tasks with no additional memory overheads. We demonstrate a speedup of up to 2.92x compared to the baseline model for image, video, and 3D shape representation and rendering tasks.

show abstract

Learning a Neural 3D Texture Space From 2D Exemplars

Cited by 84 publications

References 26 publications

Projective Urban Texturing

Projective Urban Texturing

Tm-Net

CoordX: Accelerating Implicit Neural Representation with a Split MLP Architecture

Contact Info

Product

Resources

About