OutCast: Outdoor Single‐image Relighting with Cast Shadows

Griffiths, David; Ritschel, Tobias; Philip, Julien

doi:10.1111/cgf.14467

Cited by 10 publications

(9 citation statements)

References 90 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Baselines. We compare our results to the method of [PMGD21] which is specifically designed for complete scenes, Ten- GRP22] to relight each individual rendered frame using the target direction. We trained TensoIR [JLX * 23] using the default configuration but modified the "density_shift" parameter from −10 to −8 to achieve best results on our data.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A Diffusion Approach to Radiance Field Relighting using Multi‐Illumination Synthesis

Poirier‐Ginter,

Gauthier,

Phillip

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

Relighting radiance fields is severely underconstrained for multi‐view data, which is most often captured under a single illumination condition; It is especially hard for full scenes containing multiple objects. We introduce a method to create relightable radiance fields using such single‐illumination data by exploiting priors extracted from 2D image diffusion models. We first fine‐tune a 2D diffusion model on a multi‐illumination dataset conditioned by light direction, allowing us to augment a single‐illumination capture into a realistic – but possibly inconsistent – multi‐illumination dataset from directly defined light directions. We use this augmented data to create a relightable radiance field represented by 3D Gaussian splats. To allow direct control of light direction for low‐frequency lighting, we represent appearance with a multi‐layer perceptron parameterized on light direction. To enforce multi‐view consistency and overcome inaccuracies we optimize a per‐image auxiliary feature vector. We show results on synthetic and real multi‐view data under single illumination, demonstrating that our method successfully exploits 2D diffusion model priors to allow realistic 3D relighting for complete scenes.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Qualitative comparisons are shown in Fig. 11; on the left we show the ground truth relit image rendered in Blender, and we then show our results, as well as those from Outcast [GRP22], Relightable 3D Gaussians [GGL * 23] and TensoIR [JLX * 23]. Please refer to the supplementary HTML viewer for more results.…”

Section: Discussionmentioning

confidence: 99%

A Diffusion Approach to Radiance Field Relighting using Multi‐Illumination Synthesis

Poirier‐Ginter,

Gauthier,

Phillip

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…The staggering advances in machine learning in the last decade have also had a profound effect on imagebased relighting [Debevec et al 2000], enabling new capabilities and improving quality [Bemana et al 2020;Ren et al 2015;Xu et al 2018]. Deep learning has subsequently been applied to more specialized relighting tasks for portraits Meka et al 2019;Sun et al 2019Sun et al , 2020, full bodies [Guo et al 2019;Kanamori and Endo 2018;Meka et al 2020;Yeh et al 2022;Zhang et al 2021a], and outdoor scenes [Griffiths et al 2022;Meshry et al 2019;. It is unclear how to extend these methods to handle scenes that contain objects with ill-defined shapes (e.g., fur) and translucent and specular materials.…”

Section: Related Workmentioning

confidence: 99%

Relighting Neural Radiance Fields with Shadow and Highlight Hints

Zeng

Chen

Dong

et al. 2023

Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Proceedings

View full text Add to dashboard Cite

This paper presents a novel neural implicit radiance representation for free viewpoint relighting from a small set of unstructured photographs of an object lit by a moving point light source different from the view position. We express the shape as a signed distance function modeled by a multi layer perceptron. In contrast to prior relightable implicit neural representations, we do not disentangle the different light transport components, but model both the local and global light transport at each point by a second multi layer perceptron that, in addition, to density features, the current position, the normal (from the signed distance function), view direction, and light position, also takes shadow and highlight hints to aid the network in modeling the corresponding high frequency light transport effects. These hints are provided as a suggestion, and we leave it up to the network to decide how to incorporate these in the final relit result. We demonstrate and validate our neural implicit representation on synthetic and real scenes exhibiting a wide variety of shapes, material properties, and global illumination light transport. CCS CONCEPTS• Computing methodologies → Image-based rendering; Reflectance modeling.

show abstract

“…Image relighting. Given the complexity of image relighting, prior methods mainly focus on a specific use case such as portraits [Pandey et al 2021;Yeh et al 2022] or outdoor structures [Griffiths et al 2022]. These methods rely on large-scale, difficult-to-obtain datasets, or multi-view scenes [Nicolet et al 2020;Philip et al 2019Philip et al , 2021, in order to achieve realistic results.…”

Section: Object Insertionmentioning

confidence: 99%

Intrinsic Harmonization for Illumination-Aware Image Compositing

Careaga,

Miangoleh,

Aksoy

2023

SIGGRAPH Asia 2023 Conference Papers

View full text Add to dashboard Cite

Figure 1: We propose an illumination-aware image harmonization approach for in-the-wild imagery. Our method is formulated in the intrinsic image domain. We use off-the-shelf networks to generate albedo, shading and surface normals for the input composite and background image. We first harmonize the albedo of the background and foreground by predicting image editing parameters. Using normals and shading we estimate a simple lighting model for the background illumination. With this lighting model, we render Lambertian shading for the foreground and refine it using a network trained on segmentation datasets via self-supervision. When compared to prior works we are the only method that is capable of modeling realistic lighting effects. Image credit: Unsplash users mak_jp, Daniil Silantev, Jean-

show abstract

OutCast: Outdoor Single‐image Relighting with Cast Shadows

Cited by 10 publications

References 90 publications

A Diffusion Approach to Radiance Field Relighting using Multi‐Illumination Synthesis

A Diffusion Approach to Radiance Field Relighting using Multi‐Illumination Synthesis

Relighting Neural Radiance Fields with Shadow and Highlight Hints

Intrinsic Harmonization for Illumination-Aware Image Compositing

Contact Info

Product

Resources

About