Intrinsic Decompositions for Image Editing

Abstract

Figure 1: We evaluate state-of-the-art intrinsic image decomposition algorithms based on their ability to produce seamless, artifact-free results for image edits. To fairly compare different methods, we automatize the image editing process. Left to right, by Poisson-based inpainting of the reflectance layer, we remove a logo on a shirt using the metod of Barron et al. [BM15], we add a picnic blanket over a shadow with the method of Grosse et al. [GJAF09], and add a painting over colored shadows with the method of Bousseau et al. [BPD09]. AbstractIntrinsic images are a mid-level representation of an image that decompose the image into reflectance and illumination layers. The reflectance layer captures the color/texture of surfaces in the scene, while the illumination layer captures shading effects caused by interactions between scene illumination and surface geometry. Intrinsic images have a long history in computer vision and recently in computer graphics, and have been shown to be a useful representation for tasks ranging from scene understanding and reconstruction to image editing. In this report, we review and evaluate past work on this problem. Specifically, we discuss each work in terms of the priors they impose on the intrinsic image problem. We introduce a new synthetic ground-truth dataset that we use to evaluate the validity of these priors and the performance of the methods. Finally, we evaluate the performance of the different methods in the context of image-editing applications.

Unsupervised Deep Single‐Image Intrinsic Decomposition using Illumination‐Varying Image Sequences

“…Single-image IID is the process of decomposing an image I into a dense albedo map A and shading map S. A recent survey by Bonneel et al [4] reviews and compares related work on its usability for typical CG applications. The problem is severely ambiguous: twice as many unknowns as knowns have to be estimated.…”

“…Finally, other priors restrict the range of values for S (e.g., grayscale and/or S 1, which disallows specularities). The overwhelming majority of IID works make such choices [4]. Notably, [16] defines I = AD + S p , where a diffuse D < 1 and the specular S p ∈ R + components are colored.…”

“…Shading from [4] Shading chromatic. Figure 5: Ground-truth shadings obtained with physicallybased rendering (odd rows) and its chroma-only image obtained by setting L = 50 in the CIE-Lab color space (even rows).…”

“…Intrinsic image decomposition is the inverse process in which one tries to recover intrinsic properties of a scene from a single 2D image representation. Many applications benefit from having access to this disentangled representation to allow for improved scene understanding, feature and shadow detection, stylization, relighting, object insertion and many more [4]. In this paper, we tackle the problem of separating a single image I into an albedo image A (i.e., a lighting and acquisition-independent representation of the scene) and its complementary image termed shading 1 S.…”

“…To guide an optimization procedure towards a desirable goal for this under-determined problem, one can define priors based on observation of a simplified Mondrian world (e.g., A is piecewise constant or S is smooth [21]), reduce degrees of freedom (e.g., S is greyscale [13,34,3]) and/or provide additional input data (e.g., image depth [10]). On real-world use cases however, automatic SIID remains unsolved [4]. Machine learning offers a way to learn valid priors, rather than impose them.…”