Combining Shape from Shading and Stereo: A Variational Approach for the Joint Estimation of Depth, Illumination and Albedo

Abstract

Shape from shading (SfS) and stereo are two fundamentally different strategies for imagebased 3-D reconstruction. While approaches for SfS infer the depth solely from pixel intensities, methods for stereo are based on finding correspondences across images.In this paper we propose a joint variational method that combines the advantages of both strategies. By integrating recent stereo and SfS models into a single minimisation framework, we obtain an approach that exploits shading information to improve upon the reconstruction quality of robust stereo methods. To this end, we fuse a Lambertian SfS approach with a robust stereo model and supplement the resulting energy functional with a detailpreserving anisotropic second-order smoothness term. Moreover, we extend the novel model in such a way that it jointly estimates depth, albedo and illumination. This in turn makes the approach applicable to objects with non-uniform albedo as well as to scenes with unknown illumination.

Beyond Multi-view Stereo: Shading-Reflectance Decomposition

“…In this view, our alternating scheme will be modified in order to include an additional step aiming at estimating the normals, in the spirit of the recent multi-view shape-from-shading approach presented in [9].…”

“…We argue that it may be more meaningful to keep the geometry simple, yet providing the artists with a series of 2D-maps representing reflectance and fine-scale geometry, under the form of albedo and depth maps. A first step in this direction has recently been achieved in [9], where a variational framework for the joint recovery of depth, reflectance and lighting maps is introduced. Yet, this approach relies on the choice of a single reference view, the other views being used only for the sake of stereo matching.…”

3D Reconstruction Based on Fusing Active Structured Laser and Passive Stereo Techniques

“…Pacheco et al [27], used the same techniques in [26] but, with high resolution camera and different Laser projector. Maurer et al [28], presented a compact 3D system, through the use of passive-stereo vision and active-shape from shading techniques. Bylow et al [29], also used photometric stereo and shape from shading technique but with different illumination's conditions.…”

Polarimetric Multi-View Inverse Rendering

“…However, it generally assumes a known or calibrated camera and lighting setup. Some advanced methods [39,56,57], including multiview photometric stereo [35,47] and MVIR [32], combine the two approaches to take both advantages. These methods typically estimate camera poses and an initial model based on SfM and MVS, and then refine the initial model, especially in texture-less regions, by using shading cues from multiple viewpoints.…”