Depth From a Polarisation + RGB Stereo Pair

Abstract

Estimated specular mask Corrected normal Depth from stereo (a) Setup and Input (b) Local disambiguation Estimated Albedo Estimated Shape (c) Output Section 4 Section 5 Section 6 Figure 1: Overview: From a stereo pair of one polarisation image and one RGB image (a) we merge stereo depth with polarisation normals using a higher order graphical model (b) before estimating an albedo map and the final geometry (c). AbstractIn this paper, we propose a hybrid depth imaging system in which a polarisation camera is augmented by a second image from a standard digital camera. For this modest increase in equipment complexity over conventional shapefrom-polarisation, we obtain a number of benefits that enable us to overcome longstanding problems with the polarisation shape cue. The stereo cue provides a depth map which, although coarse, is metrically accurate. This is used as a guide surface for disambiguation of the polarisation surface normal estimates using a higher order graphical model. In turn, these are used to estimate diffuse albedo. By extending a previous shape-from-polarisation method to the perspective case, we show how to compute dense, detailed maps of absolute depth, while retaining a linear formulation. We show that our hybrid method is able to recover dense 3D geometry that is superior to state-of-the-art shape-from-polarisation or two view stereo alone.

Uncalibrated, Two Source Photo-Polarimetric Stereo

“…Kadambi et al [11], [33] combine a single polarisation image with a depth map obtained by an RGBD camera. Zhu and Smith [34] add an additional non-polarised camera giving a stereo cue from which they can estimate a rough depth map. The depth map is used to disambiguate the surface normals and provide a base surface for integration.…”

Deep multimodal fusion for semantic image segmentation: A survey

“…As shown in 12, the micro-grid appears on the polarized surface and reveals an intensity change according to the polarizer affected. To tackle practical problems in computer vision, polarization images have been widely applied to object detection [160], image dehazing [161], depth estimation [162,163].…”

A Comprehensive Introduction to Photometric 3D-Reconstruction

“…Space carving [75,76] or RGBD sensors [57,58] have been employed to obtain initial 3D-shape, from which the ambiguities in SfP are resolved. Zhu et al [135] used polarisation and an RGBD stereo pair to disambiguate the polarisation surface normal estimates using a higher order graphical model. Cameras with multiple spectral bands [74,51] could be useful for disambiguating and estimating the refractive index of the surface.…”