Grasping unknown objects using an Early Cognitive Vision system for general scene understanding

Abstract

Abstract-Grasping unknown objects based on real-world visual input is a challenging problem. In this paper, we present an Early Cognitive Vision system that builds a hierarchical representation based on edge and texture information, which is a sparse but powerful description of the scene. Based on this representation we generate edge-based and surface-based grasps. The results show that the method generates successful grasps, that the edge and surface information are complimentary, and that the method can deal with more complex scenes. We furthermore present a benchmark for visual-based grasping.

VisGraB: A Benchmark for Vision-Based Grasping

“…This setup allows for extensive experimental evaluation, supporting comparison of di erent methods, while considering noise and uncertainty in the real stereo images. Our previous work used a part of the database as a proof of concept, [27]. In this paper, we present a large database along with software tools to evaluate the generated grasps.…”

“…A less restricted strategy for grasping unknown objects in the real world based on a hierarchical edge representation of the scene has been presented in [28]. In [27], this method has been extended to include surface information. Other approached apply learning methods to gain grasp experience and apply this in grasping unknown objects, for instance, based on the parameters of a superquadric representation of the object [7,26], shape context [3], or features of edge elements [2].…”

“…This means that the graspgeneration methods provided by the users of the benchmark need to be able not only to deal with the grasp-generation process but also with generating object hypotheses, if the grasp generation method requires that. Methods such as [27,28,31], works directly on images without the need to explicitly generate object hypotheses. There are also several methods that perform figure-ground segmentation at first, such as using a bottom-up segmentation method based on color and depth [30] and the additional use of a table plane detection [3] methods do not need a segmentation of the scene, because they use single image points for pinch grasps, e.g., [31].…”

“…Some of the studies have been done entirely in simulation (e.g., [6,7,12,26]), whereas others are performed in the real world (e.g., [2,28,31]). Moreover, the studies use di erent measures to evaluate the grasp performance; [12,14,26] used a measure based on the grasp wrench space [10], the time to generate the first good grasp is used in [6], and other studies grasp and lift the object in order to determine grasp success [27,30,31], or a more fine-grained grasp classification [2,28]. We use all these quality measures in our benchmark.…”

“…In Section 3, a description of how to use the benchmark is given. Next, in Section 4, we give a baseline performance for the benchmark using our method described in [27]. The paper ends with a discussion in Section 5.…”