PointNetGPD: Detecting Grasp Configurations from Point Sets

Liang, Hongzhuo; Ma, Xiaojun; Li, Shuang; Görner, Michael; Tang, Song; Fang, Bin; Sun, Fuchun; Zhang, Jianwei

doi:10.1109/icra.2019.8794435

Cited by 280 publications

(238 citation statements)

References 26 publications

Supporting

Mentioning

238

Contrasting

Order By: Relevance

“…Some early deep learning-based methods recast the grasping estimation as an object detection task, which is to predict grasp pose from the 2D images [ 33 ]. Recently, with the development of the deep learning architecture for 3D point cloud processing [ 7 , 34 ], more studies focus on grasping estimation by using the 3D data, such as Grasp Pose Detection (GPD) [ 35 ] and PointNet GPD [ 36 ]. In the agricultural cases, most of works [ 37 , 38 , 39 ] pick fruit by translating towards the targets, which cannot secure the success rate of harvesting in unstructured environments.…”

Section: Literature Reviewmentioning

confidence: 99%

Real-Time Fruit Recognition and Grasping Estimation for Robotic Apple Harvesting

Kang

Zhou

Wang

et al. 2020

Sensors

130

View full text Add to dashboard Cite

Robotic harvesting shows a promising aspect in future development of agricultural industry. However, there are many challenges which are still presented in the development of a fully functional robotic harvesting system. Vision is one of the most important keys among these challenges. Traditional vision methods always suffer from defects in accuracy, robustness, and efficiency in real implementation environments. In this work, a fully deep learning-based vision method for autonomous apple harvesting is developed and evaluated. The developed method includes a light-weight one-stage detection and segmentation network for fruit recognition and a PointNet to process the point clouds and estimate a proper approach pose for each fruit before grasping. Fruit recognition network takes raw inputs from RGB-D camera and performs fruit detection and instance segmentation on RGB images. The PointNet grasping network combines depth information and results from the fruit recognition as input and outputs the approach pose of each fruit for robotic arm execution. The developed vision method is evaluated on RGB-D image data which are collected from both laboratory and orchard environments. Robotic harvesting experiments in both indoor and outdoor conditions are also included to validate the performance of the developed harvesting system. Experimental results show that the developed vision method can perform highly efficient and accurate to guide robotic harvesting. Overall, the developed robotic harvesting system achieves 0.8 on harvesting success rate and cycle time is 6.5 s.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Real-Time Fruit Recognition and Grasping Estimation for Robotic Apple Harvesting

Kang

Zhou

Wang

et al. 2020

Sensors

130

View full text Add to dashboard Cite

show abstract

“…All of the aforementioned prior work is trained for one particular hand. Typically, this is a two-fingered gripper with exceptions including [4,[10][11][12][13][14]. [7] is the only approach we are aware of that considers two different grippers (a suction cup and a parallel jaw gripper).…”

Section: A Data-driven Grasp Synthesis For Novel Objectsmentioning

confidence: 99%

“…However, this requires to define a finite set of pre-grasp shapes which limits the dexterity of a multi-fingered robot hand, e.g. [4,[11][12][13][14]. Also the resulting grasps are typically power grasps which are great for pick-and-place.…”

Section: A Data-driven Grasp Synthesis For Novel Objectsmentioning

confidence: 99%

See 1 more Smart Citation

UniGrasp: Learning a Unified Model to Grasp With Multifingered Robotic Hands

Shao

Ferreira

Jorda

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

To achieve a successful grasp, gripper attributes including geometry and kinematics play a role equally important to the target object geometry. The majority of previous work has focused on developing grasp methods that generalize over novel object geometry but are specific to a certain robot hand. We propose UniGrasp, an efficient data-driven grasp synthesis method that considers both the object geometry and gripper attributes as inputs. UniGrasp is based on a novel deep neural network architecture that selects sets of contact points from the input point cloud of the object. The proposed model is trained on a large dataset to produce contact points that are in force closure and reachable by the robot hand. By using contact points as output, we can transfer between a diverse set of Nfingered robotic hands. Our model produces over 90% valid contact points in Top10 predictions in simulation and more than 90% successful grasps in the real world experiments for various known two-fingered and three-fingered grippers. Our model also achieves 93% and 83% successful grasps in the real world experiments for a novel two-fingered and five-fingered anthropomorphic robotic hand, respectively.

show abstract

“…The result is a permutation invariant feature vector that describes the point cloud. The PointNet architecture forms the basis for much recent work; this includes methods for evaluating grasps from raw point clouds, as seen in [31]. Here, grasp candidates were sampled based on heuristics before being evaluated using a PointNet-style network.…”

Section: Related Workmentioning

confidence: 99%

Cutting Pose Prediction from Point Clouds

Philipsen

Moeslund

2020

Sensors

View full text Add to dashboard Cite

The challenge of getting machines to understand and interact with natural objects is encountered in important areas such as medicine, agriculture, and, in our case, slaughterhouse automation. Recent breakthroughs have enabled the application of Deep Neural Networks (DNN) directly to point clouds, an efficient and natural representation of 3D objects. The potential of these methods has mostly been demonstrated for classification and segmentation tasks involving rigid man-made objects. We present a method, based on the successful PointNet architecture, for learning to regress correct tool placement from human demonstrations, using virtual reality. Our method is applied to a challenging slaughterhouse cutting task, which requires an understanding of the local geometry including the shape, size, and orientation. We propose an intermediate five-Degree of Freedom (DoF) cutting plane representation, a point and a normal vector, which eases the demonstration and learning process. A live experiment is conducted in order to unveil issues and begin to understand the required accuracy. Eleven cuts are rated by an expert, with 8 / 11 being rated as acceptable. The error on the test set is subsequently reduced through the addition of more training data and improvements to the DNN. The result is a reduction in the average translation from 1.5 cm to 0.8 cm and the orientation error from 4.59° to 4.48°. The method’s generalization capacity is assessed on a similar task from the slaughterhouse and on the very different public LINEMOD dataset for object pose estimation across view points. In both cases, the method shows promising results. Code, datasets, and other materials are available in Supplementary Materials.

show abstract

PointNetGPD: Detecting Grasp Configurations from Point Sets

Cited by 280 publications

References 26 publications

Real-Time Fruit Recognition and Grasping Estimation for Robotic Apple Harvesting

Real-Time Fruit Recognition and Grasping Estimation for Robotic Apple Harvesting

UniGrasp: Learning a Unified Model to Grasp With Multifingered Robotic Hands

Cutting Pose Prediction from Point Clouds

Contact Info

Product

Resources

About