Object modeling using a ToF camera under an uncertainty reduction approach

Abstract

Abstract-Time-of-Flight (ToF) cameras deliver 3D images at 25 fps, offering great potential for developing fast object modeling algorithms. Surprisingly, this potential has not been extensively exploited up to now. A reason for this is that, since the acquired depth images are noisy, most of the available registration algorithms are hardly applicable. A further difficulty is that the transformations between views are in general not accurately known, a circumstance that multi-view object modeling algorithms do not handle properly under noisy conditions. In this work, we take into account both uncertainty sources (in images and camera poses) to generate spatially consistent 3D object models fusing multiple views with a probabilistic approach. We propose a method to compute the covariance of the registration process, and apply an iterative state estimation method to build object models under noisy conditions.

ToF cameras for active vision in robotics

“…At closer distances, ToF cameras have been applied to object modeling [5,6], precise surface reconstruction [7], and to grasp known [8] and unknown [9] objects. We focus our review on two complementary areas: scene-related tasks and object-related tasks.…”

“…A classical solution in the area of object modeling is the use of calibrated stereo rigs. Therefore, initial works were devoted to their comparison with [37] Dynamic object detection and classification Color and light independence PMD Hussmann and Liepert [38] Object pose Easy object/background segmentation PMD Guomundsson et al [39] Known object pose estimation Light independent / Absolute scale SR3 Beder et al [40] Surface reconstruction using patchlets ToF easily combines with stereo PMD Fuchs and May [7] Precise surface reconstruction 3D at high rate SR3/O3D100 (Depth) Dellen et al [5] 3D object reconstruction 3D at high rate SR3 (Depth) Foix et al [6] Kuehnle et al [8] Object recognition for grasping 3D allow geometric primitives search SR3 Grundmann et al [41] Collision free object manipulation 3D at high rate SR3 + stereo Reiser and Kubacki [42] Position based visual servoing 3D is simply obtained / No model needed SR3 (Depth) Gachter et al [43] Object part detection for classification 3D at high rate SR3 Shin et al [44] SR2 Klank et al [45] Mobile manipulation Easy table/object segmentation SR4 Marton et al [46] Object categorization ToF easily combines with stereo SR4 + color Nakamura et al [47] Mobile manipulation Easy table segmentation SR4 + color Saxena et al [9] Grasping unknown objects 3D at high rate SR3 + stereo Zhu et al [48] Short range depth maps ToF easily combines with stereo SR3 + stereo Lindner et al [49] Object segmentation for recognition Easy color registration PMD + color camera Fischer et al [50] Occlusion handling in virtual objects 3D at high rate PMD + color camera…”

“…The resulting model is obtained after reducing noise and outliers by treating the coarse registered point cloud as a system of interacting masses connected via elastic forces. Alternatively, Foix et al [6] propose a method to compute the covariance of the point clouds registration process (ICP), and apply an iterative view-based aggregation method to build object models under noisy conditions. Their method does not need accurate hand-eye calibration since it uses globally consistent probabilistic data fusion by means of a view-based information-form SLAM algorithm, and can be executed in real time taking full advantage of the high frame rate of the ToF camera.…”

“…Figure 8 shows an example, where in the first view the segmentation algorithm, that uses depth similarity between adjusted surfaces, fails to distinguish two different leaves. Using a next-best-view algorithm [6], a new view is selected that maximizes the difference in depth of the two leaves, thus the algorithm is now capable of distinguishing the two leaves.…”

Robust surface tracking in range image sequences

“…However, the basic ICP method [17] is a pairwise matching algorithm which does not take into account past measurements [18,6], hence the error starts to propagate. The ICP algorithm has been previously combined with Kalman filtering for object reconstruction [19]. However, in this case, the background was removed, leaving only the target object.…”