Reflectance and Shape Estimation with a Light Field Camera under Natural Illumination

Ngo, Thanh Trung; Nagahara, Hajime; Nishino, Ko; Taniguchi, Rin Ichiro; Yagi, Yasushi

doi:10.5244/c.31.110

Cited by 5 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lighting estimation is a classic challenge in computer vision and computer graphics, and it is critical for realistic relighting during object insertion [18,13,27,2,28,22,26,11] and image synthesis [47,40,39,43,38,46,42,44,37]. Traditional approaches require user intervention or assumptions about the underlying illumination model, scene geometry, etc.…”

Section: Related Workmentioning

confidence: 99%

GMLight: Lighting Estimation via Geometric Distribution Approximation

Zhan,

Yu,

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

Lighting estimation from a single image is an essential yet challenging task in computer vision and computer graphics. Existing works estimate lighting by regressing representative illumination parameters or generating illumination maps directly. However, these methods often suffer from poor accuracy and generalization. This paper presents Geometric Mover's Light (GMLight), a lighting estimation framework that employs a regression network and a generative projector for effective illumination estimation. We parameterize illumination scenes in terms of the geometric light distribution, light intensity, ambient term, and auxiliary depth, and estimate them as a pure regression task. Inspired by the earth mover's distance, we design a novel geometric mover's loss to guide the accurate regression of light distribution parameters. With the estimated lighting parameters, the generative projector synthesizes panoramic illumination maps with realistic appearance and frequency. Extensive experiments show that GMLight achieves accurate illumination estimation and superior fidelity in relighting for 3D object insertion. The codes are available at GMLight.

show abstract

Section: Related Workmentioning

confidence: 99%

GMLight: Lighting Estimation via Geometric Distribution Approximation

Zhan,

Yu,

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Pioneering research in general dynamic scene reconstruction from multiple handheld widebaseline cameras (Ballan et al 2010;Taneja et al 2011) exploited prior reconstruction of the background scene to allow dynamic foreground segmentation and reconstruction. Recent work (Ngo et al 2019) estimates shape of dynamic objects from handheld cameras exploiting GANs. However these approaches either work for static/indoor scenes or exploit strong prior assumptions such as silhouette information, known background or scene structure.…”

Section: Dynamic Scene Reconstructionmentioning

confidence: 99%

Temporally Coherent General Dynamic Scene Reconstruction

et al. 2020

View full text Add to dashboard Cite

Existing techniques for dynamic scene reconstruction from multiple wide-baseline cameras primarily focus on reconstruction in controlled environments, with fixed calibrated cameras and strong prior constraints. This paper introduces a general approach to obtain a 4D representation of complex dynamic scenes from multi-view wide-baseline static or moving cameras without prior knowledge of the scene structure, appearance, or illumination. Contributions of the work are: an automatic method for initial coarse reconstruction to initialize joint estimation; sparse-to-dense temporal correspondence integrated with joint multi-view segmentation and reconstruction to introduce temporal coherence; and a general robust approach for joint segmentation refinement and dense reconstruction of dynamic scenes by introducing shape constraint. Comparison with state-of-the-art approaches on a variety of complex indoor and outdoor scenes, demonstrates improved accuracy in both multiview segmentation and dense reconstruction. This paper demonstrates unsupervised reconstruction of complete temporally coherent 4D scene models with improved non-rigid object segmentation and shape reconstruction and its application to various applications such as free-view rendering and virtual reality.

show abstract

“…Pioneering research in general dynamic scene reconstruction from multiple handheld wide-baseline cameras [5,60] exploited prior reconstruction of the background scene to allow dynamic foreground segmentation and reconstruction. Recent work [46] estimates shape of dynamic objects from handheld cameras exploiting GANs. However these approaches either work for static/indoor scenes or exploit strong prior assumptions such as silhouette information, known background or scene structure.…”

Section: Dynamic Scene Reconstructionmentioning

confidence: 99%

Temporally Coherent General Dynamic Scene Reconstruction

Mustafa¹,

Volino²,

Kim³

et al. 2019

Preprint

View full text Add to dashboard Cite

Existing techniques for dynamic scene reconstruction from multiple wide-baseline cameras primarily focus on reconstruction in controlled environments, with fixed calibrated cameras and strong prior constraints. This paper introduces a general approach to obtain a 4D representation of complex dynamic scenes from multi-view wide-baseline static or moving cameras without prior knowledge of the scene structure, appearance, or illumination. Contributions of the work are: An automatic method for initial coarse reconstruction to initialize joint estimation; Sparse-to-dense temporal correspondence integrated with joint multi-view segmentation and reconstruction to introduce temporal coherence; and a general robust approach for joint segmentation refinement and dense reconstruction of dynamic scenes by introducing shape constraint. Comparison with state-of-the-art approaches on a variety of complex indoor and outdoor scenes, demonstrates improved accuracy in both multi-view segmentation and dense reconstruction. This paper demonstrates unsupervised reconstruction of complete temporally coherent 4D scene models with improved non-rigid object segmentation and shape reconstruction and its application to free-viewpoint rendering and virtual reality.

show abstract

Reflectance and Shape Estimation with a Light Field Camera under Natural Illumination

Cited by 5 publications

References 24 publications

GMLight: Lighting Estimation via Geometric Distribution Approximation

GMLight: Lighting Estimation via Geometric Distribution Approximation

Temporally Coherent General Dynamic Scene Reconstruction

Temporally Coherent General Dynamic Scene Reconstruction

Contact Info

Product

Resources

About