End-to-End Learning to Grasp via Sampling from Object Point Clouds

Abstract

The ability to grasp objects is an essential skill that enables many robotic manipulation tasks. Recent works have studied point cloud-based methods for object grasping by starting from simulated datasets and have shown promising performance in real-world scenarios. Nevertheless, many of them still strongly rely on ad-hoc geometric heuristics to generate grasp candidates, which fail to generalize to objects with significantly different shapes with respect to those observed during training. Moreover, these methods are generally inefficient with respect to the number of training samples and the time needed during deployment. In this paper, we propose an end-to-end learning solution to generate 6-DOF parallel-jaw grasps starting from the partial view of the object. Our Learning to Grasp (L2G) method takes as input object point clouds and is guided by a principled multi-task optimization objective that generates a diverse set of grasps combining contact point sampling, grasp regression, and grasp evaluation. With a thorough experimental analysis, we show the effectiveness of the proposed method as well as its robustness and generalization abilities.

Video based basketball shooting prediction and pose suggestion system

“…Due to the high processing time to extract geometric information from the coordinates enumeration, truncating input point volume, by either downsampling [27] or target segmentation [22], is necessary. L2G [30] alternatively designs an learnable sampler, which can be jointly tuned in the end-to-end training process. It assumes properly designed sampling procedure can retain critical information for grasp synthesis, which is not always the case especially for high-resolution input.…”