Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep Learning

Abstract

Vacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex-Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When evaluated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98%, 82%, and 58% respectively, improving to 81% in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at http://berkeleyautomation.github.io/dex-net.

Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter

“…Suction Grasping: For simulation experiments, grasp planning is done with a Dex-Net 1.0 suction grasping policy [27]. For physical experiments, suction cup grasps are planned with a Dex-Net 3.0 GQ-CNN [26], with maskbased constraints to plan grasps only on the goal object's segmentation mask. The GQ-CNN evaluates each candidate grasp and returns the grasp with the highest predicted quality and its associated quality metric.…”

ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact

“…Real-world demonstrations in various forms are then used to process these samples: filter them or train a machine learning algorithm to predict success. For example, Mahler et al [1], [3], [2], [4] execute those grasps with a robot and record success/failure. Song et al [29] learn a Bayes Net to jointly model post-grasp task and discretized hand pose.…”

“…Recent work on robotic grasping of household objects focuses on using large amounts of data collected by trying random grasping actions, often using parallel-jaw or suctioncup end effectors [1], [2], [3], [4]. This approach generates lots of self-supervised data that enables training of robust grasp policies.…”

Generate What You Can’t See - a View-dependent Image Generation

“…Single-view images can be used for effective planning of grasping points for vacuum-based end effectors because only a single visible point of contact of suitable surface geometry is required [9]. Along with a greater number of fingers in a gripper, the estimation of grasping points becomes more difficult.…”