ROI-based Robotic Grasp Detection for Object Overlapping Scenes

Abstract

Grasp detection considiering the affiliations between grasps and their owner in object overlapping scenes is a necessary and challenging task for the practical use of the robotic grasping approach. In this paper, a robotic grasp detection algorithm named ROI-GD is proposed to provide a feasible solution to this problem based on Region of Interest (ROI), which is the region proposal for objects. ROI-GD uses features from ROIs to detect grasps instead of the whole scene. It has two stages: the first stage is to provide ROIs in the input image and the second-stage is the grasp detector based on ROI features. We also contribute a multi-object grasp dataset, which is much larger than Cornell Grasp Dataset, by labeling Visual Manipulation Relationship Dataset. Experimental results demonstrate that ROI-GD performs much better in object overlapping scenes and at the meantime, remains comparable with state-of-the-art grasp detection algorithms on Cornell Grasp Dataset and Jacquard Dataset. Robotic experiments demonstrate that ROI-GD can help robots grasp the target in single-object and multi-object scenes with the overall success rates of 92.5% and 83.8% respectively.

Robotic grasp detection based on image processing and random forest

“…Their work is to use the depth image to predict the grasp quality and grasp pose of every pixel. Other methods in [40] developed a roi-based detection system which can grasp objects in a pile of objects scene. All their works [3,18,34] made the full use of convolutional neural network which need no pre-processing and could automatically extract features.…”

Recalling of Multiple Grasping Methods From an Object Image With a Convolutional Neural Network

“…To solve this problem, it is desirable to learn the grasping method using the object shape without using a prior database and shape primitives. Several studies have been performed for estimating the various grasping methods and its confidence score using a neural network (NN) [4,5]. In addition, investigations have been performed to estimate the grasping method from the features of the posture and the color information of the object using the random forest classification algorithm [6,7].…”

Recalling of Multiple Grasping Methods From an Object Image With a Convolutional Neural Network

“…As a method that does not require the shape primitives, there is a method of recalling the grasping method from a realistic object shape using machine learning. Several studies have been performed for estimating the various grasping methods and its confidence score using a neural network (NN) [3,4]. In addition, investigations have been performed to estimate the grasping method from the features of the posture and the color information of the object using the random forest classification algorithm [5,6].…”