Intrinsic Decomposition of Image Sequences from Local Temporal Variations

Abstract

We present a method for intrinsic image decomposition, which aims to decompose images into reflectance and shading layers. Our input is a sequence of images with varying illumination acquired by a static camera, e.g. an indoor scene with a moving light source or an outdoor timelapse. We leverage the local color variations observed over time to infer constraints on the reflectance and solve the ill-posed image decomposition problem. In particular, we derive an adaptive local energy from the observations of each local neighborhood over time, and integrate distant pairwise constraints to enforce coherent decomposition across all surfaces with consistent shading changes. Our method is solely based on multiple observations of a Lambertian scene under varying illumination and does not require user interaction, scene geometry, or an explicit lighting model. We compare our results with several intrinsic decomposition methods on a number of synthetic and captured datasets.

Learning Intrinsic Image Decomposition from Watching the World

“…Intrinsic images from multiple observations. A number of methods, starting with Weiss [37], estimate intrinsic images from time-lapse sequences by assuming constant reflectance but varying shading over time [27,36,13,25,24]. Such an approach is similar to our training regime, although a crucial distinction is that once our model is trained, we can run it on a single image.…”

End-To-End Time-Lapse Video Synthesis From a Single Outdoor Image

“…In their work, the authors demonstrate the possibility of analyzing the dataset with automatic annotations, such as semantic labels, season changes, and weather conditions. It is also possible to extract illumination, material and geometry information from time-lapse videos as shown in previous methods [23,29,17]. Most recently, Li and Snavely [20] proposed to learn singleview intrinsic image decomposition from time-lapse videos in the wild without ground truth data.…”

Learning to Separate Multiple Illuminants in a Single Image

“…This problem is commonly solved by assuming that the reflectance of the scene is piece-wise constant while the illumination varies smoothly [4]. Several approaches build on this by further imposing priors on non-local reflectance [42,38,5], or on the consistency of reflectance for image sequences captured with varying illumination [30,21,26]. A common assumption in intrinsic image methods is that the scene is lit by a single dominant illuminant.…”