Rich intrinsic image decomposition of outdoor scenes from multiple views

Abstract

Intrinsic images aim at separating an image into its reflectance and illumination components to facilitate further analysis or manipulation. This separation is severely ill-posed and the most successful methods rely on user indications or precise geometry to resolve the ambiguities inherent to this problem. In this paper we propose a method to estimate intrinsic images from multiple views of an outdoor scene without the need for precise geometry and with a few manual steps to calibrate the input. We use multiview stereo to automatically reconstruct a 3D point cloud of the scene. Although this point cloud is sparse and incomplete, we show that it provides the necessary information to compute plausible sky and indirect illumination at each 3D point. We then introduce an optimization method to estimate sun visibility over the point cloud. This algorithm compensates for the lack of accurate geometry and allows the extraction of precise shadows in the final image. We finally propagate the information computed over the sparse point cloud to every pixel in the photograph using image-guided propagation. Our propagation not only separates reflectance from illumination, but also decomposes the illumination into a sun, sky and indirect layer. This rich decomposition allows novel image manipulations as demonstrated by our results.

Variational Reflectance Estimation from Multi-view Images

“…Instead of introducing possibly unverifiable priors, or relying on user interactions, ambiguities can be reduced by assuming that the geometry of the scene is known. Intrinsic image decomposition has for instance been addressed using an RGB-D camera [9] or, closer to our proposal, multiple views of the same scene under different angles [19,20]. In the latter works, the geometry is first extracted from the multi-view images, before the problem of reflectance estimation is addressed.…”

Multispectral Image Intrinsic Decomposition via Subspace Constraint

“…Assorted methods take advantage of additional information, including images sequences [33,22] and videos [24] to avoid shadow effect in poor lighting condition. With the improvement of sensing devices like kinect, depth cue [2,11,24] or surface normal [27] have been applied to strengthen their assumption.…”

Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields

“…These single-image based methods, however, are inherently limited by the fundamental ill-posedness of the problem. To partially alleviate this limitation, several approaches have utilized additional input, such as multiple images [26,27,28], user interaction [29,30], and measured depth maps [1,2,3,4,5]. The use of additional data such as measured depth clearly increases performance but reduces their applicability.…”