Efficient active depth sensing by laser speckle projection system

Abstract

Abstract. An active depth sensing approach by laser speckle projection system is proposed. After capturing the speckle pattern with an infrared digital camera, we extract the pure speckle pattern using a direct-global separation method. Then the pure speckles are represented by Census binary features. By evaluating the matching cost and uniqueness between the real-time image and the reference image, robust correspondences are selected as support points. After that, we build a disparity grid and propose a generative graphical model to compute disparities. An iterative approach is designed to propagate the messages between blocks and update the model. Finally, a dense depth map can be obtained by subpixel interpolation and transformation. The experimental evaluations demonstrate the effectiveness and efficiency of our approach. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Accurate depth sensing by filtering IR cost on the guide of RGB information

“…After assigning disparity to all the pixels, we use linear interpolation [13] to obtain sub-pixel disparity. Then the sub-pixel disparity d sub is converted to depth value by triangular geometry,…”

“…To be clear, we process RGB-Raw data rather than RGB-D data. The motivation that we process RGB-Raw data is that the raw data retains the original information of the IR speckle pattern [12,13], which makes it possible to recover more accurate depth with the assistance of RGB data.…”

“…Schaffer et al [9] projected a sequence of speckle pattern onto the the object, and computed depth by temporal correlation on this pattern sequence. Other researchers have tried to do depth sensing with spatial correlation technique on a pair input image [10][11][12][13]. However, because of the loss of structure information in the IR image, there are always unreliable values, especially on the object boundaries.…”

“…To find the correspondences robustly, researchers have developed many active methods to encode the scene. Among them, random speckle pattern [9][10][11][12][13] has attracted more and more attentiveness by researches these years. Schaffer et al [9] projected a sequence of speckle pattern onto the the object, and computed depth by temporal correlation on this pattern sequence.…”

Dense Image Matching With Two Steps of Expansion

“…In this paper, we introduce a method based on two steps of expansion and inspired by the seed-and-grow method (Adams and Bischof, 1994;Sharma et al, 2011;Wang et al, 2013;Yin et al, 2014). It determines the corresponding information on all the feature points of an image through two steps of expansion over a small number of seed and feature points.…”