Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards

Mahler, Jeffrey; Pokorny, Florian T.; Hou, Brian; Roderick, Melrose; Laskey, Michael; Aubry, Mathieu; Kohlhoff, Kai; Kröger, Torsten; Kuffner, James; Goldberg, Ken

doi:10.1109/icra.2016.7487342

Cited by 329 publications

(259 citation statements)

References 40 publications

Supporting

Mentioning

237

Contrasting

Order By: Relevance

“…As most approaches, in particular deep learning, are limited by its data consumption, data generation becomes a fundamental challenge. Possible solutions were supposed: First, training in simulation with subsequent simto-real transfer showed great results for grasp quality estimation [7], [8]. However, as contact forces are difficult to simulate, training of more complex object interactions for pre-grasping manipulation might be challenging.…”

Section: Related Workmentioning

confidence: 99%

Robot Learning of Shifting Objects for Grasping in Cluttered Environments

Berscheid

Meifbner

Kröger

2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Abstract-Robotic grasping in cluttered environments is often infeasible due to obstacles preventing possible grasps. Then, pre-grasping manipulation like shifting or pushing an object becomes necessary. We developed an algorithm that can learn, in addition to grasping, to shift objects in such a way that their grasp probability increases. Our research contribution is threefold: First, we present an algorithm for learning the optimal pose of manipulation primitives like clamping or shifting. Second, we learn non-prehensible actions that explicitly increase the grasping probability. Making one skill (shifting) directly dependent on another (grasping) removes the need of sparse rewards, leading to more data-efficient learning. Third, we apply a real-world solution to the industrial task of bin picking, resulting in the ability to empty bins completely. The system is trained in a self-supervised manner with around 25 000 grasp and 2500 shift actions. Our robot is able to grasp and file objects with 274 ± 3 picks per hour. Furthermore, we demonstrate the system's ability to generalize to novel objects.

show abstract

Section: Related Workmentioning

confidence: 99%

Robot Learning of Shifting Objects for Grasping in Cluttered Environments

Berscheid

Meifbner

Kröger

2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…To train UniGrasp model, we require data that consists of object point clouds annotated with sets of contact points that are in force closure and reachable by specific grippers. We generate this data set in simulation as commonly done for other data-driven approaches [4,6,7,32]. To construct this dataset, we select 1000 object models that are available in Bullet [31] and scale each object up to five different sizes to yield 3275 object instances.…”

Section: Grasp Dataset Generationmentioning

confidence: 99%

UniGrasp: Learning a Unified Model to Grasp With Multifingered Robotic Hands

Shao

Ferreira

Jorda

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

To achieve a successful grasp, gripper attributes including geometry and kinematics play a role equally important to the target object geometry. The majority of previous work has focused on developing grasp methods that generalize over novel object geometry but are specific to a certain robot hand. We propose UniGrasp, an efficient data-driven grasp synthesis method that considers both the object geometry and gripper attributes as inputs. UniGrasp is based on a novel deep neural network architecture that selects sets of contact points from the input point cloud of the object. The proposed model is trained on a large dataset to produce contact points that are in force closure and reachable by the robot hand. By using contact points as output, we can transfer between a diverse set of Nfingered robotic hands. Our model produces over 90% valid contact points in Top10 predictions in simulation and more than 90% successful grasps in the real world experiments for various known two-fingered and three-fingered grippers. Our model also achieves 93% and 83% successful grasps in the real world experiments for a novel two-fingered and five-fingered anthropomorphic robotic hand, respectively.

show abstract

“…The grasp labels in the Dex-net 2.0 dataset were acquired via random sampling of antipodal grasp candidates. A heuristic-based approach developed in previous work (Dex-Net 1.0 [28]) was used to compute a robustness metric. This metric was thresholded to determine the grasp robustness label, i.e.…”

Section: B Dex-net 20 Datasetmentioning

confidence: 99%

GQ-STN: Optimizing One-Shot Grasp Detection based on Robustness Classifier

Gariépy¹,

Ruel²,

Chaib-draa³

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Grasping is a fundamental robotic task needed for the deployment of household robots or furthering warehouse automation. However, few approaches are able to perform grasp detection in real time (frame rate). To this effect, we present Grasp Quality Spatial Transformer Network (GQ-STN), a oneshot grasp detection network. Being based on the Spatial Transformer Network (STN), it produces not only a grasp configuration, but also directly outputs a depth image centered at this configuration. By connecting our architecture to an externally-trained grasp robustness evaluation network, we can train efficiently to satisfy a robustness metric via the backpropagation of the gradient emanating from the evaluation network. This removes the difficulty of training detection networks on sparsely annotated databases, a common issue in grasping. We further propose to use this robustness classifier to compare approaches, being more reliable than the traditional rectangle metric. Our GQ-STN is able to detect robust grasps on the depth images of the Dex-Net 2.0 dataset with 92.4 % accuracy in a single pass of the network. We finally demonstrate in a physical benchmark that our method can propose robust grasps more often than previous sampling-based methods, while being more than 60 times faster.

show abstract

Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards

Cited by 329 publications

References 40 publications

Robot Learning of Shifting Objects for Grasping in Cluttered Environments

Robot Learning of Shifting Objects for Grasping in Cluttered Environments

UniGrasp: Learning a Unified Model to Grasp With Multifingered Robotic Hands

GQ-STN: Optimizing One-Shot Grasp Detection based on Robustness Classifier

Contact Info

Product

Resources

About