Robotic Grasping of Novel Objects using Vision

Abstract

We consider the problem of grasping novel objects, specifically ones that are being seen for the first time through vision. Grasping a previously unknown object, one for which a 3-d model is not available, is a challenging problem. Further, even if given a model, one still has to decide where to grasp the object. We present a learning algorithm that neither requires, nor tries to build, a 3-d model of the object. Given two (or more) images of an object, our algorithm attempts to identify a few points in each image corresponding to good locations at which to grasp the object. This sparse set of points is then triangulated to obtain a 3-d location at which to attempt a grasp. This is in contrast to standard dense stereo, which tries to triangulate every single point in an image (and often fails to return a good 3-d model). Our algorithm for identifying grasp locations from an image is trained via supervised learning, using synthetic images for the training set. We demonstrate this approach on two robotic manipulation platforms. Our algorithm successfully grasps a wide variety of objects, such as plates, tape-rolls, jugs, cellphones, keys, screwdrivers, staplers, a thick coil of wire, a strangely shaped power horn, and others, none of which were seen in the training set. We also apply our method to the task of unloading items from dishwashers.

Patch Based Synthesis for Single Depth Image Super-Resolution

“…Some synthetic depth datasets also exist, e.g. [35], but they typically contain single objects. Due to the large amount of redundancy in depth scenes (e.g.…”

An overview of 3D object grasp synthesis algorithms

“…Some approaches associate grasp parameters or hand shapes to object geometric features in order to find good grasps in terms of stability [61,62]. Other techniques learn to identify grasping regions in an object image [63,64]. These techniques are discussed in the following.…”

“…A learning approach for robotic grasping of novel objects is also presented by Saxena et al [63]. Based on the idea that there are certain visual features that indicate ''good'' grasps, and that remain consistent across many different objects (such as coffee mugs handles or long objects such as pens that can be grasped at their mid-point), a learning approach that uses these visual features was proposed to predict ''good'' grasping points.…”

Reconstruction and Verification of 3D Object Models for Grasping

“…A computer vision and machine learning based method is used in [16] to train classifiers that can predict the grasping points in an image. This is then applied to images of unseen objects.…”