Learning to Separate Multiple Illuminants in a Single Image

Zhuo, Hui; Chakrabarti, Ayan; Sunkavalli, Kalyan; Sankaranarayanan, Aswin C.

doi:10.1109/cvpr.2019.00390

Cited by 10 publications

(3 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Traditional color constancy methods remove illumination color casts in images and estimate the colors of objects under the canonical illuminant, assuming that the image is taken under uniform color illuminant [13,32]. Source separation, an extension of color constancy, separates an image lit by multiple illuminants into multiple images lit by each illuminant in the scene [5,14,15]. Although the solutions for these tasks do not mention the light position or direction, they are applicable for general objects and lights.…”

Section: Related Workmentioning

confidence: 99%

Single-Shot Multi-light-Direction Searching on Discretized Lighting Space

2021

View full text Add to dashboard Cite

Existing methods for light direction estimation solve the problem only by presupposing certain assumptions, e.g., that only one light illuminates the object, the object has a uniform texture or smoothly varying surface, or there is a known object in the scene. However, these methods cannot be used reliably in real-world settings. We propose a framework that can be used in natural settings to estimate multi-light directions from a single image. To tackle this novel and challenging problem, we divide the lighting space into ranges and formulate the estimation problem as a problem of searching the ranges where the light is cast from a multi-illuminant image. We propose a two-step approach. First, we generate single-illuminant images, one of which assumes that there is only light on one range. Next, we select some images from the candidates that could each be a component of a multi-illuminant image under physical constraints. The experimental results demonstrated that the proposed method outperformed the direct-estimation method when the lighting space is finely divided.

show abstract

Section: Related Workmentioning

confidence: 99%

Single-Shot Multi-light-Direction Searching on Discretized Lighting Space

2021

View full text Add to dashboard Cite

show abstract

“…Both statistics based methods [14,43] and deep learning based methods [6,18,30] have used a uniform illumination model to solve the CC task. More recent methods focus on the multi-illuminant CC problem, working on pairs of images taken under different illuminants [19,21] or using two-branch networks to describe local illuminants [39]. Our white-balancing network (WBNet) is a single image color constancy model targeted to document images.…”

Section: Related Workmentioning

confidence: 99%

Intrinsic Decomposition of Document Images In-the-Wild

Das,

Sial,

et al. 2020

Preprint

View full text Add to dashboard Cite

Automatic document content processing is affected by artifacts caused by the shape of the paper, non-uniform and diverse color of lighting conditions. Fully-supervised methods on real data are impossible due to the large amount of data needed. Hence, the current state of the art deep learning models are trained on fully or partially synthetic images. However, document shadow or shading removal results still suffer because: (a) prior methods rely on uniformity of local color statistics, which limit their application on real-scenarios with complex document shapes and textures and; (b) synthetic or hybrid datasets with non-realistic, simulated lighting conditions are used to train the models. In this paper we tackle these problems with our two main contributions. First, a physically constrained learning-based method that directly estimates document reflectance based on intrinsic image formation which generalizes to challenging illumination conditions. Second, a new dataset that clearly improves previous synthetic ones, by adding a large range of realistic shading and diverse multi-illuminant conditions, uniquely customized to deal with documents in-the-wild. The proposed architecture works in two steps. First, a white balancing module neutralizes the color of the illumination on the input image. Based on the proposed multi-illuminant dataset we achieve a good white-balancing in really difficult conditions. Second, the shading separation module accurately disentangles the shading and paper material in a self-supervised manner where only the synthetic texture is used as a weak training signal (obviating the need for very costly ground truth with disentangled versions of shading and reflectance). The proposed approach leads to significant generalization of document reflectance estimation in real scenes with challenging illumination. We extensively evaluate on the real benchmark datasets available for intrinsic image decomposition and document shadow removal tasks. Our reflectance estimation scheme, when used as a pre-processing step of an OCR pipeline, shows a 26% improvement of character error rate (CER), thus, proving the practical applicability. The data and code will be available at: https://github.com/cvlab-stonybrook/DocIIW.

show abstract

“…Unrolled networks [29,30,31,32,33], PDE-Nets [34], and variational networks [35,36] can all be classified into this category. During training, auxiliary intermediate losses can be inserted to guarantee the learned parameters indeed carry their corresponding physical meanings as well [37,38,39].…”

Section: Categorizing Prior Work In Physics-based Learningmentioning

confidence: 99%

Blending Diverse Physical Priors with Neural Networks

Ba,

Zhao,

Kadambi

2019

Preprint

View full text Add to dashboard Cite

Machine learning in context of physical systems merits a re-examination of the learning strategy. In addition to data, one can leverage a vast library of physical prior models (e.g. kinematics, fluid flow, etc) to perform more robust inference. The nascent sub-field of physics-based learning (PBL) studies the blending of neural networks with physical priors. While previous PBL algorithms have been applied successfully to specific tasks, it is hard to generalize existing PBL methods to a wide range of physics-based problems. Such generalization would require an architecture that can adapt to variations in the correctness of the physics, or in the quality of training data. No such architecture exists. In this paper, we aim to generalize PBL, by making a first attempt to bring neural architecture search (NAS) to the realm of PBL. We introduce a new method known as physics-based neural architecture search (PhysicsNAS) that is a top-performer across a diverse range of quality in the physical model and the dataset. Code is available at: https://github.com/PhysicsNAS/PhysicsNAS.

show abstract

Learning to Separate Multiple Illuminants in a Single Image

Cited by 10 publications

References 40 publications

Single-Shot Multi-light-Direction Searching on Discretized Lighting Space

Single-Shot Multi-light-Direction Searching on Discretized Lighting Space

Intrinsic Decomposition of Document Images In-the-Wild

Blending Diverse Physical Priors with Neural Networks

Contact Info

Product

Resources

About