Learning 6-DOF Grasping Interaction via Deep Geometry-Aware 3D Representations

Abstract

This paper focuses on the problem of learning 6-DOF grasping with a parallel jaw gripper in simulation. Our key idea is constraining and regularizing grasping interaction learning through 3D geometry prediction. We introduce a deep geometry-aware grasping network (DGGN) that decomposes the learning into two steps. First, we learn to build mental geometry-aware representation by reconstructing the scene (i.e., 3D occupancy grid) from RGBD input via generative 3D shape modeling. Second, we learn to predict grasping outcome with its internal geometry-aware representation. The learned outcome prediction model is used to sequentially propose grasping solutions via analysis-by-synthesis optimization. Our contributions are fourfold: (1) To best of our knowledge, we are presenting for the first time a method to learn a 6-DOF grasping net from RGBD input; (2) We build a grasping dataset from demonstrations in virtual reality with rich sensory and interaction annotations. This dataset includes 101 everyday objects spread across 7 categories, additionally, we propose a data augmentation strategy for effective learning; (3) We demonstrate that the learned geometry-aware representation leads to about 10% relative performance improvement over the baseline CNN on grasping objects from our dataset. (4) We further demonstrate that the model generalizes to novel viewpoints and object instances.

Generating Grasp Poses for a High-DOF Gripper Using Neural Networks

“…For example, a grasp pose should not have penetration with O. In prior methods [36], [10], [24], the neural network is not responsible for ensuring the quality of the grasp poses, but we can guarantee highquality grasp poses when solving Equation 1 after training. However, in our case, the neural network is used to generate x directly, so our final results are very sensitive to the outputs of the neural network.…”

“…There are several avenues for future work. One is to consider an end-to-end architecture that predicts grasp poses directly from multi-view depth images, similar to [36]. Another direction is to consider more topologically complex target objects, such as high-genus models.…”

Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity

“…A comparison of various control interfaces shows that general purpose hardware is deficient while special purpose hardware is more accurate but is not widely available [19,26]. Virtual reality-based free-space controllers have recently been proposed both for data collection [23,39] and policy learning [40,43]. While these methods have shown the utility of data, they do not provide a seamlessly scalable data collection mechanism.…”

Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics

“…The density of the point cloud is compared to the densities of the real clouds to match it accordingly. The densities are computed by randomly sampling 1 10 of the points and averaging the distances to their nearest neighbors. The raw partial point clouds have a Hausdorff distance of 8.7 millimeters with respect to the full ground truth mugs, while our predicted mugs have a distance of 3.8 millimeters.…”