Seeing the World in a Bag of Chips

Park, Jeong Joon; Holynski, Aleksander; Seitz, Steven M.

doi:10.1109/cvpr42600.2020.00149

Cited by 32 publications

(17 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most methods that recover factorized full 3D models for relighting and view synthesis rely on additional observations instead of strong priors. A common strategy is to use 3D geometry obtained from active scanning [Guo et al 2019;Lensch et al 2003;Park et al 2020;Schmitt et al 2020;Zhang et al 2020], proxy models Dong et al 2014;Gao et al 2020;Georgoulis et al 2015;Sato et al 2003], silhouette masks [Godard et al 2015;Oxholm and Nishino 2014;Xia et al 2016], or multi-view stereo (followed by surface reconstruction and meshing) [Goel et al 2020;Laffont et al 2012;Nam et al 2018;] as a starting point before recovering reflectance and refined geometry. In this work, we show that starting with geometry estimated using a state-of-the-art neural volumetric representation enables us to recover a fully-factorized 3D model just using images captured under one illumination, without requiring any additional observations.…”

Section: Related Workmentioning

confidence: 99%

NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination

Zhang,

Srinivasan,

Deng

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination

Zhang,

Srinivasan,

Deng

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Geometry adjustment is modeled from an initial depth fusion through a linear combination of surface normals, which can inflate or deflate the surface. Park et al [31] model interreflection and Fresnel reflectance in their learning-based recovery of scene properties from RGBD imagery, via surface light field and specular reflectance map reconstructions. Both approaches as-sume accurate geometry initialization, while we include results on reconstructions from coarser initializations.…”

Section: Geometry and Materials Reconstructionmentioning

confidence: 99%

Shape From Tracing: Towards Reconstructing 3D Object Geometry and SVBRDF Material from Images via Differentiable Path Tracing

Goel¹,

Cohen²,

Guesman³

et al. 2020

Preprint

View full text Add to dashboard Cite

Reconstructing object geometry and material from multiple views typically requires optimization. Differentiable path tracing is an appealing framework as it can reproduce complex appearance effects. However, it is difficult to use due to high computational cost. In this paper, we explore how to use differentiable ray tracing to refine an initial coarse mesh and per-mesh-facet material representation. In simulation, we find that it is possible to reconstruct fine geometric and material detail from low resolution input views, allowing high-quality reconstructions in a few hours despite the expense of path tracing. The reconstructions successfully disambiguate shading, shadow, and global illumination effects such as diffuse interreflection from material properties. We demonstrate the impact of different geometry initializations, including space carving, multi-view stereo, and 3D neural networks. Finally, with input captured using smartphone video and a consumer 360 • camera for lighting estimation, we also show how to refine initial reconstructions of real-world objects in unconstrained environments.

show abstract

“…Specular highlights occur when the light source and the direction of the viewer are halfway between the normal to the surface. In most cases, specular highlights are nuisances creating undesired artifacts in the image, however, they can also be useful for determining the direction of the light source within the environment in such cases as photometric stereo or surface light field reconstruction [1].…”

Section: Introductionmentioning

confidence: 99%

“…as SunCG [5] lack sufficient model quality to recover specular highlights. Therefore, we propose the LIGHTS Dataset 1 , constructed from high-quality architectural 3D models with variation in lighting design and rendering parameters to create near photo-realistic scenes (see Fig. 1).…”

Section: Introductionmentioning

confidence: 99%

Lights: Light Specularity Dataset For Specular Detection In Multi-View

Elkhouly

Tsesmelis

Bue

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Specular highlights are commonplace in images, however, methods for detecting them and removing the phenomenon are particularly challenging. A reason for this is the difficulty in creating a dataset for training or evaluation, as in the real world, we lack the necessary control over the environment. Therefore, we propose a novel physically-based rendered LIGHT Specularity (LIGHTS) Dataset for the evaluation of the specular highlight detection task. Our dataset consists of 18 high-quality architectural scenes, where each scene is rendered with multiple views. In total, the dataset contains 2, 603 views with an average of 145 views per scene. Additionally, we propose a simple aggregation based method for specular highlight detection that outperforms prior work by 3.6% in two orders of magnitude less time on our dataset.

show abstract

Seeing the World in a Bag of Chips

Cited by 32 publications

References 63 publications

NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination

NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination

Shape From Tracing: Towards Reconstructing 3D Object Geometry and SVBRDF Material from Images via Differentiable Path Tracing

Lights: Light Specularity Dataset For Specular Detection In Multi-View

Contact Info

Product

Resources

About