Transferring Experience from Simulation to the Real World for Precise Pick-And-Place Tasks in Highly Cluttered Scenes

Kleeberger, Kilian; Völk, Markus; Moosmann, Marius; Erik, Thiessenhusen,; Roth, Florian; Bormann, Richard; Huber, Marco F.

doi:10.1109/iros45743.2020.9341709

Cited by 13 publications

(10 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Performance would likely be improved using more accurate robotic grippers and transfer to specific scenes or grippers, e.g. [28]. However, this work intends to minimise extensive retraining to reinforce the generalisability of the model.…”

Section: Modelmentioning

confidence: 99%

Evaluating Gaussian Grasp Maps for Generative Grasping Models

Prew¹,

Breckon²,

Bordewich³

et al. 2022

Preprint

View full text Add to dashboard Cite

Generalising robotic grasping to previously unseen objects is a key task in general robotic manipulation. The current method for training many antipodal generative grasping models rely on a binary ground truth grasp map generated from the centre thirds of correctly labelled grasp rectangles. However, these binary maps do not accurately reflect the positions in which a robotic arm can correctly grasp a given object. We propose a continuous Gaussian representation of annotated grasps to generate ground truth training data which achieves a higher success rate on a simulated robotic grasping benchmark. Three modern generative grasping networks are trained with either binary or Gaussian grasp maps, along with recent advancements from the robotic grasping literature, such as discretisation of grasp angles into bins and an attentional loss function. Despite negligible difference according to the standard rectangle metric, Gaussian maps better reproduce the training data and therefore improve success rates when tested on the same simulated robot arm by avoiding collisions with the object: achieving 87.94% accuracy. Furthermore, the best performing model is shown to operate with a high success rate when transferred to a real robotic arm, at high inference speeds, without the need for transfer learning. The system is then shown to be capable of performing grasps on an antagonistic physical object dataset benchmark.

show abstract

Section: Modelmentioning

confidence: 99%

Evaluating Gaussian Grasp Maps for Generative Grasping Models

Prew¹,

Breckon²,

Bordewich³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Fig. 5 shows the results of all possible combinations for a fixed point for the real object named IPABar [7] of the Fraunhofer IPA Bin-Picking dataset [41,7]. Point combinations not excluded by the comparison to the stroke of the gripper can then be checked for their contact angles.…”

Section: Grasp Synthesismentioning

confidence: 99%

“…All described steps are shown schematically in Fig. 7 and on the sample object IPABar [7] in Fig. 8.…”

Section: Grasp Clusteringmentioning

confidence: 99%

“…Numerous work places today still operate manually and their poor ergonomics or basic and monotonous tasks still reveal existing obstacles for a fully automated factory. Even a seemingly simple task such as grasping and retrieving an object from a bin with chaotically stored objects remains a major problem in robotics [2,6,7,8]. To enable an increased efficiency in handling tasks, the so-called bin-picking problem needs to be solved.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Grasp Pose Generation for Parallel Jaw Grippers

Kleeberger¹,

Roth²,

Bormann³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper presents a novel approach for the automatic offline grasp pose synthesis on known rigid objects for parallel jaw grippers. We use several criteria such as gripper stroke, surface friction, and a collision check to determine suitable 6D grasp poses on an object. In contrast to most available approaches, we neither aim for the best grasp pose nor for as many grasp poses as possible, but for a highly diverse set of grasps distributed all along the object. In order to accomplish this objective, we employ a clustering algorithm to the sampled set of grasps. This allows to simultaneously reduce the set of grasp pose candidates and maintain a high variance in terms of position and orientation between the individual grasps. We demonstrate that the grasps generated by our method can be successfully used in real-world robotic grasping applications.

show abstract

“…Object Sets and Photorealism: Motivated by recent progress in the field of dexterous grasping based on RGBD data [16], we find it beneficial to categorize existing work into depth only or photo-realistic data. While depth only datasets [17], [4], [18], [19], [20], [21] are sufficient for training state-of-the-art grasp detection networks such as [17], [22], [4], they are not applicable to recent multi-modal sensor fusion approaches [16], [3], [23], [24]. Besides this limitation, order picking systems usually depend on an additional upstream object detection.…”

Section: Introductionmentioning

confidence: 99%

MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis

Gilles¹,

Chen²,

Winter³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous bin picking poses significant challenges to vision-driven robotic systems given the complexity of the problem, ranging from various sensor modalities, to highly entangled object layouts, to diverse item properties and gripper types. Existing methods often address the problem from one perspective. Diverse items and complex bin scenes require diverse picking strategies together with advanced reasoning. As such, to build robust and effective machine-learning algorithms for solving this complex task requires significant amounts of comprehensive and high quality data. Collecting such data in real world would be too expensive and time prohibitive and therefore intractable from a scalability perspective. To tackle this big, diverse data problem, we take inspiration from the recent rise in the concept of metaverses, and introduce MetaGraspNet, a large-scale photo-realistic bin picking dataset constructed via physics-based metaverse synthesis. The proposed dataset contains 217k RGBD images across 82 different article types, with full annotations for object detection, amodal perception, keypoint detection, manipulation order and ambidextrous grasp labels for a parallel-jaw and vacuum gripper. We also provide a real dataset consisting of over 2.3k fully annotated high-quality RGBD images, divided into 5 levels of difficulties and an unseen object set to evaluate different object and layout properties. Finally, we conduct extensive experiments showing that our proposed vacuum seal model and synthetic dataset achieves state-of-the-art performance and generalizes to real world use-cases.

show abstract

Transferring Experience from Simulation to the Real World for Precise Pick-And-Place Tasks in Highly Cluttered Scenes

Cited by 13 publications

References 34 publications

Evaluating Gaussian Grasp Maps for Generative Grasping Models

Evaluating Gaussian Grasp Maps for Generative Grasping Models

Automatic Grasp Pose Generation for Parallel Jaw Grippers

MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis

Contact Info

Product

Resources

About