Real-time 3D segmentation of cluttered scenes for robot grasping

Abstract

Abstract-We present a real-time algorithm that segments unstructured and highly cluttered scenes. The algorithm robustly separates objects of unknown shape in congested scenes of stacked and partially occluded objects. The model-free approach finds smooth surface patches, using a depth image from a Kinect camera, which are subsequently combined to form highly probable object hypotheses. The real-time capabilities and the quality of the algorithm are evaluated on a benchmark database. Advantages compared to existing approaches as well as weaknesses are discussed. We also report on an autonomous grasping experiment with the Shadow Robot Hand which employs the estimated shape and pose of objects given by our algorithm in a task in which it cleans a table.

Object Partitioning Using Local Convexity

“…In addition to these qualitative results, we also provide quantitative results which compare to state-of-the-art methods on the NYU Indoor Dataset [22] and Object Segmentation Database [20]. We compare segments against ground truth using three standard measures: Weighted Overlap (WOv), which is a summary measure proposed by Silberman et al [22], as well as false negative (f n) and false positive (f p) scores from [24] and over-(F os ) and under-segmentation (F us ) from [20].…”

“…This is also true for the boundary between an object and the supporting surface. As a consequence, objects that have [20] RGB-D + Texture + Geometry -----4.5% 7.9% Uckermann et al [24] NO LEARNING -1.9% 3.3% 7.8% 7.3% -- Table 1. Comparison of different segmentation methods on the OSD dataset using weighted overlap WOv (the higher, the better), false positives fp, false negatives fn, as well as over-and under-segmentation Fos and Fus (the lower, the better).…”

“…As inspiration for a general rule for breaking scenes into elemental parts, we look to psychophysical studies, mostly performed on 2D images, which suggest that the transition between convex and concave image parts might be indicative of the separation between objects and/or their parts [13,25,21,15,7,4]. While this feature has been used in machine vision to some degree [10,17,20,24,11] success has remained limited and more recent studies were forced to combine this feature with additional, often very complex feature constellations to achieve good scene partitioning [20,24,11]. It, thus, appears that direct transfer from 2D to 3D of the conventional, geometrically-defined convex-concave transition criterion shown in Fig.…”

Grasping and Manipulation of Unknown Objects Based on Visual and Tactile Feedback

“…This information is utilized on the one hand to chose the grasp prototype and on the other hand to setup an appropriate approaching controller, utilizing the symmetries inherent to all recognized object shapes. A video illustrating the segmentation capabilities and the achieved grasping skills is available at youtube [22].…”

“…To chose an appropriate grasp for a given object, we employ a real-time, model-free scene segmentation method [21], which yields individual point clouds for all objects within the scene. Into each point cloud, a superquadrics model is fitted that captures the coarse shape of the object, smoothly varying between sphere, ellipsoid, cylinder, and box [22]. This model provides an estimation of the position and orientation as well as the coarse size and shape of the object.…”