Efficient grasping from RGBD images: Learning using a new rectangle representation

Abstract

Abstract-Given an image and an aligned depth map of an object, our goal is to estimate the full 7-dimensional gripper configuration-its 3D location, 3D orientation and the gripper opening width. Recently, learning algorithms have been successfully applied to grasp novel objects-ones not seen by the robot before. While these approaches use low-dimensional representations such as a 'grasping point' or a 'pair of points' that are perhaps easier to learn, they only partly represent the gripper configuration and hence are sub-optimal.We propose to learn a new 'grasping rectangle' representation: an oriented rectangle in the image plane. It takes into account the location, the orientation as well as the gripper opening width. However, inference with such a representation is computationally expensive. In this work, we present a two step process in which the first step prunes the search space efficiently using certain features that are fast to compute. For the remaining few cases, the second step uses advanced features to accurately select a good grasp. In our extensive experiments, we show that our robot successfully uses our algorithm to pick up a variety of novel objects.

Relational Kernel-Based Grasping with Numerical Features

“…Previous works on visual-dependent robot grasping have shown promising results on learning grasping points from image-based 2D descriptors [18,26]. Other works exploit combinations of image-based and point cloud-based features [2,12]. Saxena et.…”

“…al. [12] extend this approach by computing grasping stability features from the point clouds. In their method, the point cloud features are linked to the gripper configuration, while the image-based features are linked to the visual graspability of a point.…”

Review of Deep Learning Methods in Robotic Grasp Detection

“…A major drawback of such point defined grasps, however, was that it only determined where to grasp an object and it did not determine how wide the gripper had to be opened or the required orientation for the gripper to successfully grasp the object. As a way to overcome this limitation, another popular grasp representation that has been proposed is the oriented rectangle representation that was used in [10,[18][19][20]30,31]. According to Jiang et al [30], their grasping configuration has a seven-dimensional representation containing the information of a Grasping point, Grasping orientation, and Gripper opening width.…”

“…As a way to overcome this limitation, another popular grasp representation that has been proposed is the oriented rectangle representation that was used in [10,[18][19][20]30,31]. According to Jiang et al [30], their grasping configuration has a seven-dimensional representation containing the information of a Grasping point, Grasping orientation, and Gripper opening width. In world coordinates, their grasp representation, G, is stated as G = (x, y, z, α, β, γ, l).…”

Scan segments matching for pairwise 3D alignment

“…These sensors have contributed to a wealth of research in a variety of domains including autonomous driving [17], 3D modelling [12], grasping and manipulation [21], [13] and object recognition [11], to only mention a few. The work presented here belongs to this overall effort attempting to exploit the richness of 3D data to build more accurate perception systems.…”