Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF From a Single Image

Li, Zhengqin; Shafiei, Mohammad; Ramamoorthi, Ravi; Sunkavalli, Kalyan; Chandraker, Manmohan

doi:10.1109/cvpr42600.2020.00255

Cited by 197 publications

(211 citation statements)

References 36 publications

Supporting

Mentioning

207

Contrasting

Order By: Relevance

“…Recent advances in Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) have been applied to texture synthesis problems [12], [14], [15], [16], [17], [18], [19] showing unprecedented levels of realism and quality, however, the output of these methods is not tileable. Despite recent methods [13], [16], [20], [21], [22], [23] addressing the problem of tileable texture synthesis, we show that they either assume a particular level of regularity or the generated textures lose a significant amount of visual fidelity with respect to the input exemplars. Further, most of these methods have only focused on synthesizing single images.…”

Section: Introductionmentioning

confidence: 74%

“…Moritz et al [13] propose a non-parametric approach that is able to synthesize textures from a single example while preserving its stationarity, which measures how tileable the texture is. Li et al [22] propose a GraphCuts-based algorithm. They first find a patch that optimally represents the texture, then use graph cuts to transform its borders to improve its tileability.…”

Section: Texture Tileabilitymentioning

confidence: 99%

See 1 more Smart Citation

SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps

Rodriguez-Pardo,

Garces

2022

Preprint

View full text Add to dashboard Cite

Real-time graphics applications require high-quality textured materials to convey realism in virtual environments. Generating these textures is challenging as they need to be visually realistic, seamlessly tileable, and have a small impact on the memory consumption of the application. For this reason, they are often created manually by skilled artists. In this work, we present SeamlessGAN, a method capable of automatically generating tileable texture maps from a single input exemplar. In contrast to most existing methods, focused solely on solving the synthesis problem, our work tackles both problems, synthesis and tileability, simultaneously. Our key idea is to realize that tiling a latent space within a generative network trained using adversarial expansion techniques produces outputs with continuity at the seam intersection that can then be turned into tileable images by cropping the central area. Since not every value of the latent space is valid to produce high-quality outputs, we leverage the discriminator as a perceptual error metric capable of identifying artifact-free textures during a sampling process. Further, in contrast to previous work on deep texture synthesis, our model is designed and optimized to work with multi-layered texture representations, enabling textures composed of multiple maps such as albedo, normals, etc. We extensively test our design choices for the network architecture, loss function, and sampling parameters. We show qualitatively and quantitatively that our approach outperforms previous methods and works for textures of different types.

show abstract

Section: Introductionmentioning

confidence: 74%

Section: Texture Tileabilitymentioning

confidence: 99%

SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps

Rodriguez-Pardo,

Garces

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In this section, we will review some recent lighting models and estimation methods proposed for outdoor scenes. There are also some studies dedicated for indoor lighting estimation [14][15][16][17][18][19] which is out of the scope of this paper.…”

Section: Related Workmentioning

confidence: 99%

Dual attention autoencoder for all-weather outdoor lighting estimation

Guo

et al. 2021

Sci. China Inf. Sci.

View full text Add to dashboard Cite

“…The recent learning-based works aim to estimate lighting from images by regressing representation parameters [3,59,60] or generating illumination maps [61,62].…”

Section: Lighting Estimationmentioning

confidence: 99%

“…Lighting estimation has been tackled through direct generation of illumination maps [61,62,65] and regression of parameters of representative illumination functions such as spherical harmonics function [2,59] and spherical Gaussian function [3,60]. However, the functional representation methods struggle to regress accurate frequency information (especially high-frequency information) that often leads to inaccurate shading and shadow effects in relighting [2] or requires complex optimization steps [3].…”

Section: Introductionmentioning

confidence: 99%

Image synthesis in visual machine learning

Zhan¹

View full text Add to dashboard Cite

Image Synthesis For Accurate Detection and Recognition of Texts in Scenes. European Conference on Computer Vision (ECCV), 2018. The contributions of the co-authors are as follows: • I and Shijian Lu proposed the idea, designed the experiments, and prepared the manuscript. • Chuhui Xue and I conducted the experiments. • The manuscript was revised by Shijian Lu. Fangneng Zhan, Hongyuan Zhu, and Shijian Lu. Spatial Fusion GAN for Image Synthesis. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. The contributions of the co-authors are as follows: • I and Shijian Lu proposed the idea, designed and conducted the experiments, and prepared the manuscript. • The manuscript was revised by Hongyuan Zhu and Shijian Lu.

show abstract

Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF From a Single Image

Cited by 197 publications

References 36 publications

SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps

SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps

Dual attention autoencoder for all-weather outdoor lighting estimation

Image synthesis in visual machine learning

Contact Info

Product

Resources

About