Leveraging Multi-View Image Sets for Unsupervised Intrinsic Image Decomposition and Highlight Separation

Yi, Renjiao; Tan, Ping; Lin, Stephen

doi:10.1609/aaai.v34i07.6961

Cited by 25 publications

(15 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our evaluative framework, we conduct a comprehensive comparison between our proposed Semi-TSGN and several state-of-the-art highlight removal methods, namely Tan [44], Yang [45], Shen [24], Akashi [23], Shi [27], Yi [46], Huang [47], and Fu [5] using the SHIQ/RD/PSD dataset. To the best of our knowledge, we first propose a Semi-supervised learning paradigm for highlight removal.…”

Section: Comparison With the State-of-the-artsmentioning

confidence: 99%

Semi-TSGAN: Semi-Supervised Learning for Highlight Removal Based on Teacher-Student Generative Adversarial Network

Zheng,

Yan,

Jiang

2024

Sensors

View full text Add to dashboard Cite

Despite recent notable advancements in highlight image restoration techniques, the dearth of annotated data and the lightweight deployment of highlight removal networks pose significant impediments to further advancements in the field. In this paper, to the best of our knowledge, we first propose a semi-supervised learning paradigm for highlight removal, merging the fusion version of a teacher–student model and a generative adversarial network, featuring a lightweight network architecture. Initially, we establish a dependable repository to house optimal predictions as pseudo ground truth through empirical analyses guided by the most reliable No-Reference Image Quality Assessment (NR-IQA) method. This method serves to assess rigorously the quality of model predictions. Subsequently, addressing concerns regarding confirmation bias, we integrate contrastive regularization into the framework to curtail the risk of overfitting on inaccurate labels. Finally, we introduce a comprehensive feature aggregation module and an extensive attention mechanism within the generative network, considering a balance between network performance and computational efficiency. Our experimental evaluations encompass comprehensive assessments on both full-reference and non-reference highlight benchmarks. The results demonstrate conclusively the substantive quantitative and qualitative enhancements achieved by our proposed algorithm in comparison to state-of-the-art methodologies.

show abstract

Section: Comparison With the State-of-the-artsmentioning

confidence: 99%

Semi-TSGAN: Semi-Supervised Learning for Highlight Removal Based on Teacher-Student Generative Adversarial Network

Zheng,

Yan,

Jiang

2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Commonly used priors include piece‐wise constant albedos [CZL18,LS18a,LS18b,MCZ*18,LBP*12], or sparsity of extracted albedo values [MSZ*21, GMLMG12]. A few works exploit data‐driven priors instead of hand‐crafted priors [BBS14, ZKE15, SGK*19, LSR*20, PEL*21, YTL20], which can be subject to domain discrepancy. IBL‐NeRF takes inspiration from the aforementioned prior works using single images, and adds constraints in the image space.…”

Section: Related Workmentioning

confidence: 99%

IBL‐NeRF: Image‐Based Lighting Formulation of Neural Radiance Fields

Choi,

Kim,

Kim

2023

Computer Graphics Forum

View full text Add to dashboard Cite

We propose IBL‐NeRF, which decomposes the neural radiance fields (NeRF) of large‐scale indoor scenes into intrinsic components. Recent approaches further decompose the baked radiance of the implicit volume into intrinsic components such that one can partially approximate the rendering equation. However, they are limited to representing isolated objects with a shared environment lighting, and suffer from computational burden to aggregate rays with Monte Carlo integration. In contrast, our prefiltered radiance field extends the original NeRF formulation to capture the spatial variation of lighting within the scene volume, in addition to surface properties. Specifically, the scenes of diverse materials are decomposed into intrinsic components for rendering, namely, albedo, roughness, surface normal, irradiance, and prefiltered radiance. All of the components are inferred as neural images from MLP, which can model large‐scale general scenes. Especially the prefiltered radiance effectively models the volumetric light field, and captures spatial variation beyond a single environment light. The prefiltering aggregates rays in a set of predefined neighborhood sizes such that we can replace the costly Monte Carlo integration of global illumination with a simple query from a neural image. By adopting NeRF, our approach inherits superior visual quality and multi‐view consistency for synthesized images as well as the intrinsic components. We demonstrate the performance on scenes with complex object layouts and light configurations, which could not be processed in any of the previous works.

show abstract

“…Although the intrinsic image decomposition has been extended to I = R⊗L +S (Lambertian shading L, reflectance R, and specularity S) recently [10], [11], [26], it decomposes an image into two components (R and L) by ignoring specularity for simplicity in many previous works [27], [28], [29], [30]. Therefore, intrinsic image decomposition in this paper, means separation into reflectance and illumination.…”

Section: Related Work a Intrinsic Image Decompositionmentioning

confidence: 99%

“…The original intrinsic model often approximates the reflection component to diffuse reflection, and neglects specular reflection. Recently, it is extended by considering the specular component as an additive residue term [10], [11]. The extended model first removes highlight, and is followed by the conventional intrinsic decomposition.…”

Section: Introductionmentioning

confidence: 99%

Joint Deep Estimation of Intrinsic and Dichromatic Image Decomposition

Lee

Kim

2023

IEEE Access

View full text Add to dashboard Cite

This paper proposes an image formation model that jointly combines dichromatic and intrinsic image decomposition models. The two decomposition models analyze image formation process from a different perspective, and they can be combined synergistically. It is confirmed that the proposed method performs better than the individual decomposition. The joint estimation and the study of the decomposition order ('intrinsic + dichromatic' or 'dichromatic + intrinsic') are the first attempt to the best of our knowledge. It was confirmed that the proposed 'intrinsic + dichromatic' is more optimal through experimental evaluations. We also exploit the temporal property of AC light sources, which can further improve the decomposition performance. The experimental results show that the proposed model can make an accurate image decomposition and achieve a remarkable color constancy performance.INDEX TERMS Intrinsic image decomposition, dichromatic model, color constancy, AC light, high-speed video.

show abstract

Leveraging Multi-View Image Sets for Unsupervised Intrinsic Image Decomposition and Highlight Separation

Cited by 25 publications

References 41 publications

Semi-TSGAN: Semi-Supervised Learning for Highlight Removal Based on Teacher-Student Generative Adversarial Network

Semi-TSGAN: Semi-Supervised Learning for Highlight Removal Based on Teacher-Student Generative Adversarial Network

IBL‐NeRF: Image‐Based Lighting Formulation of Neural Radiance Fields

Joint Deep Estimation of Intrinsic and Dichromatic Image Decomposition

Contact Info

Product

Resources

About