3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations

Yin, Kangxue; Gao, Jun; Shugrina, Maria; Khamis, Sameh; Fidler, Sanja

doi:10.1109/iccv48922.2021.01223

Cited by 52 publications

(13 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6.4. Yin et al [44] recently proposed a geometry and texture stylization approach that is optimization based and uses differentiable rendering, a fundamentally different approach to operating on texture map data than the one we explore in this work.…”

Section: Spherical Images Superpixels and Texture Mapsmentioning

confidence: 99%

See 1 more Smart Citation

SelectionConv: Convolutional Neural Networks for Non-rectilinear Image Data

Hart¹,

Whitney²,

Morse³

2022

Preprint

View full text Add to dashboard Cite

Convolutional Neural Networks have revolutionized visionapplications. There are image domains and representations, however, that cannot be handled by standard CNNs (e.g., spherical images, superpixels). Such data are usually processed using networks and algorithms specialized for each type. In this work, we show that it may not always be necessary to use specialized neural networks to operate on such spaces. Instead, we introduce a new structured graph convolution operator that can copy 2D convolution weights, transferring the capabilities of already trained traditional CNNs to our new graph network. This network can then operate on any data that can be represented as a positional graph. By converting non-rectilinear data to a graph, we can apply these convolutions on these irregular image domains without requiring training on large domain-specific datasets. Results of transferring pre-trained image networks for segmentation, stylization, and depth prediction are demonstrated for a variety of such data forms.

show abstract

Section: Spherical Images Superpixels and Texture Mapsmentioning

confidence: 99%

“…Others have attempted style transfer between two different 3D objects [11,35,44], but we are not aware of other work attempting direct style transfer between the texture map of the 3D mesh and a 2D image.…”

Section: Masked Image and 3d Mesh Style Transfermentioning

confidence: 99%

SelectionConv: Convolutional Neural Networks for Non-rectilinear Image Data

Hart¹,

Whitney²,

Morse³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Lifting style transfer into 3D has been explored for texturing individual objects [32,42,55] or faces [24]. However, they focus on isolated objects (not room-scale scenes) and do not utilize 3D data.…”

Section: D Style Transfermentioning

confidence: 99%

“…Thus, we require a pose set covering most of the scene to optimize the texture completely. In contrast, stylizing in texture space directly [55] is problematic for a room-scale texture parametrization, which may contain many seams.…”

Section: Texture Optimizationmentioning

confidence: 99%

StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions

Höllein¹,

Johnson²,

Nießner³

2021

Preprint

View full text Add to dashboard Cite

Figure 1. We perform style transfer on reconstructed 3D meshes by synthesizing stylized textures. We compute style transfer losses on views of the scene and backpropagate gradients to the texture. Depth and surface normal data from the mesh enable 3D-aware stylization, preventing artifacts that arise from standard 2D losses. Our stylized meshes can be rendered using the traditional graphics pipeline.

show abstract

“…T He shape correspondence learning problem is fundamental to geometry processing and computer vision and has been used as a key component in many downstream applications such as deformation modeling [1], texture mapping [2], and medical imaging [3], to name a few.…”

Section: Introductionmentioning

confidence: 99%

Learning Canonical Embeddings for Unsupervised Shape Correspondence with Locally Linear Transformations

He¹

2022

Preprint

View full text Add to dashboard Cite

<p>We present a new approach to unsupervised shape correspondence learning between pairs of point clouds. We make the first attempt to adapt the classical locally linear embedding algorithm (LLE)---originally designed for nonlinear dimensionality reduction---for shape correspondence. The key idea is to find dense correspondences between shapes by first obtaining high-dimensional neighborhood-preserving embeddings of low-dimensional point clouds and subsequently aligning the source and target embeddings using locally linear transformations. We demonstrate that learning the embedding using a new LLE-inspired point cloud reconstruction objective results in accurate shape correspondences. More specifically, the approach comprises an end-to-end learnable framework of extracting high-dimensional neighborhood-preserving embeddings, estimating locally linear transformations in the embedding space, and reconstructing shapes via divergence measure-based alignment of probabilistic density functions built over reconstructed and target shapes. Our approach enforces embeddings of shapes in correspondence to lie in the same universal/canonical embedding space, which eventually helps regularize the learning process and leads to a simple nearest neighbors approach between shape embeddings for finding reliable correspondences. Comprehensive experiments show that the new method makes noticeable improvements over state-of-the-art approaches on standard shape correspondence benchmark datasets covering both human and nonhuman shapes.</p>

show abstract

3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations

Cited by 52 publications

References 41 publications

SelectionConv: Convolutional Neural Networks for Non-rectilinear Image Data

SelectionConv: Convolutional Neural Networks for Non-rectilinear Image Data

StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions

Learning Canonical Embeddings for Unsupervised Shape Correspondence with Locally Linear Transformations

Contact Info

Product

Resources

About