MetaGrasp: Data Efficient Grasping by Affordance Interpreter Network

Abstract

Data-driven approach for grasping shows significant advance recently. But these approaches usually require much training data. To increase the efficiency of grasping data collection, this paper presents a novel grasp training system including the whole pipeline from data collection to model inference. The system can collect effective grasp sample with a corrective strategy assisted by antipodal grasp rule, and we design an affordance interpreter network to predict pixelwise grasp affordance map. We define graspability, ungraspability and background as grasp affordances. The key advantage of our system is that the pixel-level affordance interpreter network trained with only a small number of grasp samples under antipodal rule can achieve significant performance on totally unseen objects and backgrounds. The training sample is only collected in simulation. Extensive qualitative and quantitative experiments demonstrate the accuracy and robustness of our proposed approach. In the real-world grasp experiments, we achieve a grasp success rate of 93% on a set of household items and 91% on a set of adversarial items with only about 6,300 simulated samples. We also achieve 87% accuracy in clutter scenario. Although the model is trained using only RGB image, when changing the background textures, it also performs well and can achieve even 94% accuracy on the set of adversarial objects, which outperforms current state-of-the-art methods.

Cloud robotic grasping of Gaussian mixture model based on point cloud projection under occlusion

“…The initial research on the gripping quality of robots was mostly based on force closure (Faverjon and Ponce, 1993) and shape closure (Diziolu and Lakshiminarayana, 1984) Jiang et al , (2011) uses a directional rectangle to represent the grasping configuration, including the position, direction and width of the gripper. After the rise of deep learning, Redmon and Angelova (2014); Morrison et al (2018); Cai et al (2019) proposed the method of using a neural network to get the representation of object-grabbing rectangle. However, the way of deep learning needs training in advance and the quality of grasping depends largely on the marked data sets.…”

“…In recent years, deep learning network has made great achievements in computer vision and other tasks such as semantic segmentation (Kaiming et al , 2017) and object detection (Bochkovskiy et al , 2020). The deep convolution network is also applied to the recognition of robot grasping posture (Redmon and Angelova, 2014; Lenz et al , 2015; Morrison et al , 2018; Cai et al , 2019). However, the training of the network needs a large number of data sets and when the network is applied to the new data that has not appeared, it often brings some problems.…”

Visual Reconstruction and Localization-Based Robust Robotic 6-DoF Grasping in the Wild

“…With the development of deep learning, a large number of methods apply object detection networks and depth sensor to obtain 3D bounding box of targets for locating the worksapce [10,11,12]. Recently, Cai et al [13] constructed an end-to-end network to esitimate the pixel level contact point of the target for perceiving the grasping location. However, above methods are limited to the preset location of workspace.…”

Memory Efficient Grasping Point Detection of Nontrivial Objects

“…Another pixel-wise affordance-based method, which provided not only grasping points, but general surfaces for robotic manipulation, was introduced in [36]. Lastly, authors in [37] proposed an affordance interpreter network to predict a pixelwise affordance map. Contrary to the above-mentioned methods, this map provided several regions with various data about grasping (stable horizontal grasp points, negative grasp locations, background, etc.).…”