GQ-STN: Optimizing One-Shot Grasp Detection based on Robustness Classifier

Abstract

Grasping is a fundamental robotic task needed for the deployment of household robots or furthering warehouse automation. However, few approaches are able to perform grasp detection in real time (frame rate). To this effect, we present Grasp Quality Spatial Transformer Network (GQ-STN), a oneshot grasp detection network. Being based on the Spatial Transformer Network (STN), it produces not only a grasp configuration, but also directly outputs a depth image centered at this configuration. By connecting our architecture to an externally-trained grasp robustness evaluation network, we can train efficiently to satisfy a robustness metric via the backpropagation of the gradient emanating from the evaluation network. This removes the difficulty of training detection networks on sparsely annotated databases, a common issue in grasping. We further propose to use this robustness classifier to compare approaches, being more reliable than the traditional rectangle metric. Our GQ-STN is able to detect robust grasps on the depth images of the Dex-Net 2.0 dataset with 92.4 % accuracy in a single pass of the network. We finally demonstrate in a physical benchmark that our method can propose robust grasps more often than previous sampling-based methods, while being more than 60 times faster.

Reconfigurable Grasp Planning Pipeline with Grasp Synthesis and Selection Applied to Picking Operations in Aerospace Factories

“…Nowadays, and driven by the appearance of new processing technologies and computer techniques, a new category of robotic grasping systems has emerged: the experience-based. In this field, several authors investigate the integration of optimization techniques into analytical analyses [9,10] and ML methods to reach grasping results that can be summarized in supervised learning (structured labeled datasets [11][12][13][14][15][16][17][18][19][20][21][22][23] or Learning by Demonstration (LbD) [24][25][26][27]) and Reinforcement Learning (RL) [28,29].…”

Double-Dot Network for Antipodal Grasp Detection

“…This method shows promising results, but suffers from a low running speed. Gariépy et al [30] train an STN (Spatial Transformer Network) to accelerate sampling; however, STN is supervised by GQ-CNN, which is not accurate enough, thus limiting the performance.…”

Depth Image–Based Deep Learning of Grasp Planning for Textureless Planar-Faced Objects in Vision-Guided Robotic Bin-Picking

“…Unlike two-stage methods, a one-stage method [28][29][30][31][32][33][34][35][36][37], namely, one-shot grasp detection, directly regresses grasp points and their classes without object segmentation or pose estimation. This method is preferable for object picking in a warehouse for two reasons.…”

“…Faced with this challenge, recent works [30][31][32][33][34][35] have trained CNN models by data generated in simulations. Josh et al [30] used domain randomization and generative models to predict scores for grasp candidates, but only tested their method on single-object grasping.…”

“…They generated grasp points in a simulator and evaluated the robustness of each using a metric named Robust Ferrari-Canny. Alexandre et al [34] extended this method to a Grasp Quality Spatial Transformer Network (GQ-STN), which improves the precision of grasp robustness predictions. Ulrich et al [35] proposed a CNN model that uses simulated depth images to predict the remaining cost-to-go of moving to the nearest viable grasp, but execution times were so long that each grasp attempt took 20-30 s.…”