Polarization imaging reflectometry in the wild

Rivière, Jérémy; Reshetouski, Ilya; Filipi, Luka; Ghosh, Abhijeet

doi:10.1145/3130800.3130894

Cited by 61 publications

(29 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A number of assumptions have been proposed to reduce ambiguity when only a few measurements of the material are available. Common priors include spatial and angular homogeneity [Zickler et al 2006], repetitive or random texture-like behavior [Ai ala et al 2016[Ai ala et al , 2015, sparse environment lighting [Dong et al 2014;Lombardi and Nishino 2016], polarization of sky lighting [Riviere et al 2017], mixture of basis BRDFs [Dong et al 2010;Hui et al 2017;Ren et al 2011], optimal sampling directions [Xu et al 2016], and user-provided constraints [Dong et al 2011]. However, many of these assumptions restrict the family of materials that can be captured.…”

Section: Related Workmentioning

confidence: 99%

Single-image SVBRDF capture with a rendering-aware deep network

Aittala²,

et al. 2018

View full text Add to dashboard Cite

Fig. 1. From a single flash photograph of a material sample (insets), our deep learning approach predicts a spatially-varying BRDF. See supplemental materials for animations with a moving light.Texture, highlights, and shading are some of many visual cues that allow humans to perceive material appearance in single pictures. Yet, recovering spatially-varying bi-directional re ectance distribution functions (SVBRDFs) from a single image based on such cues has challenged researchers in computer graphics for decades. We tackle lightweight appearance capture by training a deep neural network to automatically extract and make sense of these visual cues. Once trained, our network is capable of recovering per-pixel normal, di use albedo, specular albedo and specular roughness from a single picture of a at surface lit by a hand-held ash. We achieve this goal by introducing several innovations on training data acquisition and network design. For training, we leverage a large dataset of artist-created, procedural SVBRDFs which we sample and render under multiple lighting directions. We further amplify the data by material mixing to cover a wide diversity of shading e ects, which allows our network to work across many material classes. Motivated by the observation that distant regions of a material sample o en o er complementary visual cues, we design a network that combines an encoder-decoder convolutional track for local feature extraction with a fully-connected track for global feature extraction and propagation. Many important material e ects are view-dependent, and as such ambiguous when observed in a single image. We tackle this challenge by de ning the loss as a di erentiable SVBRDF similarity metric that compares the renderings of the predicted maps against renderings of the ground truth from several lighting and viewing directions. Combined together, these novel ingredients bring clear improvement over state of the art methods for single-shot capture of spatially varying BRDFs.is is the author's version of the work. It is posted here for your personal use. Not for redistribution. e de nitive version was published in ACM Trans.

show abstract

Section: Related Workmentioning

confidence: 99%

Single-image SVBRDF capture with a rendering-aware deep network

Aittala²,

et al. 2018

View full text Add to dashboard Cite

show abstract

“…However, the building cost of such systems is too high to make the acquisition process not available for casual users to have access to this acquisition process. To resolve this issue, practical methods using a single camera have been introduced [AWL15, RPG16, HSL*17, RRFG17, WZ15, WWZ16, SWK19, PNS18, NLGK18]. These methods can capture material appearance by inferring diffuse and specular appearance parameters from multiple observations with different view/light angles.…”

Section: Related Workmentioning

confidence: 99%

“…In contrast, we were motivated to devise a practical acquisition solution without requiring any specialized hardware setup, such as a mechanical gantry with two robotic arms or a multiview camera‐light stage. To this end, we decided to make use of a conventional RGBD camera for our acquisition setup, following the trend of state‐of‐the‐art practical techniques [AWL15, RPG16, HSL*17, RRFG17, WZ15, WWZ16, PNS18, NLGK18].…”

Section: Introductionmentioning

confidence: 99%

Progressive Acquisition of SVBRDF and Shape in Motion

Baek

Nam

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

To estimate appearance parameters, traditional SVBRDF acquisition methods require multiple input images to be captured with various angles of light and camera, followed by a post‐processing step. For this reason, subjects have been limited to static scenes, or a multiview system is required to capture dynamic objects. In this paper, we propose a simultaneous acquisition method of SVBRDF and shape allowing us to capture the material appearance of deformable objects in motion using a single RGBD camera. To do so, we progressively integrate photometric samples of surfaces in motion in a volumetric data structure with a deformation graph. Then, building upon recent advances of fusion‐based methods, we estimate SVBRDF parameters in motion. We make use of a conventional RGBD camera that consists of the colour and infrared cameras with active infrared illumination. The colour camera is used for capturing diffuse properties, and the infrared camera‐illumination module is employed for estimating specular properties by means of active illumination. Our joint optimization yields complete material appearance parameters. We demonstrate the effectiveness of our method with extensive evaluation on both synthetic and real data that include various deformable objects of specular and diffuse appearance.

show abstract

“…Specular highlights can be removed by a similar approach [ 16 ], which estimates the polarization angle of the specular highlight by measuring linear Stokes parameters. Recently, Riviere et al [ 17 ] have employed similar linear polarization measurements under uncontrolled outdoor illumination for estimating surface reflectance of planar surfaces. However, these approaches require multiple shots with rotating polarizers at a specific angle in front of an imaging device to measure the Stokes parameters.…”

Section: Related Workmentioning

confidence: 99%

Polarized Light Field Imaging for Single-Shot Reflectance Separation

Kim

Ghosh

2018

Sensors

Self Cite

View full text Add to dashboard Cite

We present a novel computational photography technique for single-shot separation of diffuse/specular reflectance, as well as novel angular domain separation of layered reflectance. We present two imaging solutions for this purpose: two-way polarized light-field (TPLF) imaging and four-way polarized light-field (FPLF) imaging. TPLF imaging consists of a polarized light-field camera, which simultaneously captures two orthogonal states of polarization. A single photograph of a subject acquired with the TPLF camera under polarized illumination then enables standard separation of diffuse (depolarizing) and polarization preserving specular reflectance using light-field sampling. We further demonstrate that the acquired data also enable novel angular separation of layered reflectance including separation of specular reflectance and single scattering in the polarization preserving component, as well as separation of shallow scattering from deep scattering in the depolarizing component. FPLF imaging further generalized the functionality of TPLF imaging under uncontrolled unpolarized or partially polarized illumination such as outdoors. We apply our approach for efficient acquisition of facial reflectance including diffuse and specular normal maps and novel separation of photometric normals into layered reflectance normals for layered facial renderings. We validate our proposed single-shot layered reflectance separation under various imaging conditions and demonstrate it to be comparable to an existing multi-shot technique that relies on structured lighting while achieving separation results under a variety of illumination conditions.

show abstract

Polarization imaging reflectometry in the wild

Cited by 61 publications

References 59 publications

Single-image SVBRDF capture with a rendering-aware deep network

Single-image SVBRDF capture with a rendering-aware deep network

Progressive Acquisition of SVBRDF and Shape in Motion

Polarized Light Field Imaging for Single-Shot Reflectance Separation

Contact Info

Product

Resources

About