Deep Learning-Based Illumination Estimation Using Light Source Classification

Abstract

Color constancy is one of the key steps in the process of image formation in digital cameras. Its goal is to process the image so that there is no influence of illumination color on the colors of objects and surfaces. To capture the target scene colors as accurately as possible, it is crucial to estimate the illumination vector with high accuracy. Unfortunately, the illumination estimation is an ill-posed problem, and solving it most often relies on assumptions. To date, various assumptions have been proposed, which resulted in a wide variety of illumination estimation methods. Statistics-based methods have shown to be appropriate for hardware implementation, but learning-based methods achieve state-of-the-art results, especially those that use deep neural networks. The large learning capacities and generalization abilities of deep neural networks can be used to develop the illumination estimation methods, which are more general and precise. This approach avoids introducing many new assumptions, which often only work in some specific situations. In this paper, a new method for illumination estimation based on light source classification is proposed. In the first step, the set of possible illuminations is reduced by classifying the input image in one of three classes. The classes include images captured in outdoor scenes under natural illuminations, images captured in outdoor scenes under artificial illuminations, and images captured in indoor scenes under artificial illuminations. In the second step, a deep illumination estimation network, which is trained exclusively on images in the class that was predicted in the first step, is applied to the input image. Dividing the illumination space into smaller regions makes the training of illumination estimation networks simpler because the distribution of image scenes and illuminations is less diverse. The experiments on the Cube+ image dataset have shown the median illumination estimation error of 1.27 • , which is an improvement of more than 25% compared to the use of the single network for all illuminations.

Iterative Convolutional Neural Network-Based Illumination Estimation

“…The average execution time on the test set using only one core was 2.04 seconds per input image. The proposed model has 14,716,227 weights which is less compared to deep learning-based illumination estimations methods evaluated on the Cube+ dataset in [19], [20], [22], which all use VGG16 network structure for feature extraction, but have more complex additional layer structures, such as attention blocks, or have multiple instances of the same network structure with different weights.…”

“…Content-based convolutional neural networks that combine weighted local illumination estimations have been proposed in [18]- [20]. In [21], [22], illumination estimation was cast into a deep learning classification problem. In [23], from an image, two illuminations were estimated using one convolutional neural network, and then using another convolutional neural network, a more probable one was chosen.…”

HSV Color-Space-Based Automated Object Localization for Robot Grasping without Prior Knowledge

“…To minimize external environmental changes, deep learning algorithms such as R-CNN [36], Fast R-CNN [37], and YOLO [38] have attracted attention in recent years [39][40][41]. However, deep-learning-based object detection has many constraints, such as a large amount of training and test data, a long learning time, an annotation task, and a large amount of calculation.…”

Illuminant Estimation Using Adaptive Neuro-Fuzzy Inference System

“…In the meantime, the eight unitary methods, i.e., GW, WP, SOG, GE1, GE2, GGW, PCAbased, and LSR, are adopted to estimate illuminant color for the input image I in . Then all these estimates are normalized and formulated into an IIF vector, θ(I in ), as described in Equation (15).…”

“…Statistics-based methods are based on some statistical features that are kept consistent in the image captured under canonical light conditions, e.g., Gray world (GW) [4,5], White patch (WP) [6], Shades of gray (SoG) [7], Gray edge (GE) [8], etc. Learning-based algorithms have a learning phase and use various image information, normally low-level image features, to pre-train models to estimate the illuminant color, e.g., natural image statistics [9], classification-based algorithm selection [10], deep learning based methods [11][12][13][14][15][16], etc. The first two types of algorithms, called unitary algorithms, use a single strategy, and usually can not balance among implementation cost, computation efforts, and algorithm complexity.…”