Adaptive Light Estimation using Dynamic Filtering for Diverse Lighting Conditions

Zhao, Junhong; Chalmers, Andrew; Rhee, Taehyun

doi:10.1109/tvcg.2021.3106497

Cited by 18 publications

(6 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the fidelity of the relighting results generated by different methods are subjectively evaluated by a user study. Results of (Gardner et al 2017(Gardner et al , 2019Chalmers et al 2020;Zhao, Chalmers, and Rhee 2021) were provided by the authors, and the method of (Zhan et al 2021) was implemented by ourselves.…”

Section: Methodsmentioning

confidence: 99%

“…(Gardner et al 2019) uses simplified spherical Gaussian model, lacking of the details of the scene, thus produces unrealistic rendering with low roughness material. Without the illumination guidance, (Chalmers et al 2020;Zhao, Chalmers, and Rhee 2021) directly predict blurry environment maps and result in unrealistic specular reflections. Our method can predicts more accurate light directions and low-frequency lighting than (Zhan et al 2021), with the Transformer-based SG regression module and the guidance of irradiance map, which produces plausible and realistic renderings with accurate shades and shadows.…”

Section: Qualitative Evaluationmentioning

confidence: 99%

“…We compare our framework with four state-of-the-art methods that directly generate environment maps (Gardner et al 2017;Chalmers et al 2020;Zhao, Chalmers, and Rhee 2021) or estimate representative illumination functions (Gardner et al 2019). We render 300 objects with SVBRDF textures using the environment maps predicted from the test set, and show relighting errors of each method in Table 1.…”

Section: Quantitative Evaluation and User Studymentioning

confidence: 99%

“…Meanwhile, this simplified lighting parametric model can not restore the details of the scene except for main light sources , which are vital to render objects with high specular reflections, thus gets low scores in the user study. In (Gardner et al 2017;Chalmers et al 2020;Zhao, Chalmers, and Rhee 2021), direct generation of environment maps without any guidance tends to overfit to the training data and generate blurry results. In contrast, our method regresses SG coefficients for main light sources, SH coefficients for low-frequency lighting, then uses a generative network to predict high-quality environment maps under the guidance of the estimated lighting parameters.…”

Section: Quantitative Evaluation and User Studymentioning

confidence: 99%

See 3 more Smart Citations

Rendering-Aware HDR Environment Map Prediction from a Single Image

Zuo

Zhang

et al. 2022

AAAI

View full text Add to dashboard Cite

High dynamic range (HDR) illumination estimation from a single low dynamic range (LDR) image is a significant task in computer vision, graphics, and augmented reality. We present a two-stage deep learning-based method to predict an HDR environment map from a single narrow field-of-view LDR image. We first learn a hybrid parametric representation that sufficiently covers high- and low-frequency illumination components in the environment. Taking the estimated illuminations as guidance, we build a generative adversarial network to synthesize an HDR environment map that enables realistic rendering effects. We specifically consider the rendering effect by supervising the networks using rendering losses in both stages, on the predicted environment map as well as the hybrid illumination representation. Quantitative and qualitative experiments demonstrate that our approach achieves lower relighting errors for virtual object insertion and is preferred by users compared to state-of-the-art methods.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Qualitative Evaluationmentioning

confidence: 99%

Section: Quantitative Evaluation and User Studymentioning

confidence: 99%

Section: Quantitative Evaluation and User Studymentioning

confidence: 99%

See 2 more Smart Citations

Rendering-Aware HDR Environment Map Prediction from a Single Image

Zuo

Zhang

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…Some researchers focus on omnidirectional view-synthesis from 360°image sequences to provide 6-DOF immersive experiences by explicitly [6] or implicitly [2] reconstructing 3D geometry. Zhao et al [59] and Xu et al [51] proposed to use convolutional networks to predict 360°HDR images for a better illumination effect when inserting virtual objects into a target scene. But they are not designed for manipulating stereo image content.…”

Section: °Image Analysis and Processingmentioning

confidence: 99%

360° Stereo Image Composition with Depth Adaption

Huang¹,

Zhang²,

Zhao³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>360° images and videos have become an economic and popular way to provide VR experiences using real-world content. However, the manipulation of the stereo panoramic content remains less explored. In this paper, we focus on the 360° image composition problem, and develop a solution that can take an object from a stereo image pair and insert it at a given 3D position in a target stereo panorama, with well-preserved geometry information. Our method uses recovered 3D point clouds to guide the composited image generation. More specifically, we observe that using only a one-off operation to insert objects into equirectangular images will never produce satisfactory depth perception and generate ghost artifacts when users are watching the result from different view directions. Therefore, we propose a novel view-dependent projection method that segments the object in 3D spherical space with the stereo camera pair facing in that direction. A deep depth densification network is proposed to generate depth guidance for the stereo image generation of each view segment according to the desired position and pose of the inserted object. We finally merge the synthesized view segments and blend the objects into the target stereo 360° scene. A user study demonstrates that our method can provide good depth perception and removes ghost artifacts. The view-dependent solution is a potential paradigm for other content manipulation methods for 360° images and videos. </p>

show abstract

Deep 360$$^\circ $$ Optical Flow Estimation Based on Multi-projection Fusion

Barnes

Huang

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Adaptive Light Estimation using Dynamic Filtering for Diverse Lighting Conditions

Cited by 18 publications

References 35 publications

Rendering-Aware HDR Environment Map Prediction from a Single Image

Rendering-Aware HDR Environment Map Prediction from a Single Image

360° Stereo Image Composition with Depth Adaption

Deep 360$$^\circ $$ Optical Flow Estimation Based on Multi-projection Fusion

Contact Info

Product

Resources

About