Dex-Net MM: Deep Grasping for Surface Decluttering with a Low-Precision Mobile Manipulator

Staub, Benno; Tanwani, Ajay Kumar; Mahler, Jeffrey; Breyer, Michel; Laskey, Michael; Takaoka, Yutaka; Bajracharya, Max; Siegwart, Roland; Goldberg, Ken

doi:10.1109/coase.2019.8842901

Cited by 5 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, there are no approaches that use a CAD-based ANN for grasp planning. Consequently, we want to develop one and evaluate its performance by comparing it with point cloud based ANN approaches such as (Staub et al, 2019). While we expect that improvements in regards to ESP should be marginal, the computation time is likely to be reduced.…”

Section: Future Workmentioning

confidence: 99%

CAD-based Grasp and Motion Planning for Process Automation in Fused Deposition Modelling

Wiedholz

Heider

Nordsieck

et al. 2021

Proceedings of the 18th International Conference on Informatics in Control, Automation and Robotics

View full text Add to dashboard Cite

Planning the right grasp pose and motion into it has been a problem in the robotic community for more than 20 years. This paper presents a model-based approach for a Pick action of a robot that increases the automation of FDM based additive manufacturing by removing a produced object from the build plate. We treat grasp pose planning, motion planning and simulation-based verification as separate components to allow a high exchangeability. When testing a variety of different object geometries, feasible grasps and motions were obtained for all objects. We also found that the computation time is highly dependent on the random seed, leading us to employ a system of budgeted runs for which we report the estimated success probability and expected running time. Within the budget, some objects never found feasible picks. Thus, we rotated these objects by 90 • which lead to a substantial improvement in success probabilities.

show abstract

Section: Future Workmentioning

confidence: 99%

CAD-based Grasp and Motion Planning for Process Automation in Fused Deposition Modelling

Wiedholz

Heider

Nordsieck

et al. 2021

Proceedings of the 18th International Conference on Informatics in Control, Automation and Robotics

View full text Add to dashboard Cite

show abstract

“…For the implementation of a highly flexible dataset on decluttering surfaces (e.g. homes and machine shops), Staub et al [212] modified Dex-Net 4.0 to generate the Dex-Net MM grasp planner for coping with the parameters of the mobile manipulator because this task can be executed using a mobile manipulator rather than a stationary industrial manipulator; hence, mobile robots were equipped with low-precision sensors and actuators alike. In a surface decluttering experiment where objects were randomly selected from 40 common machine shop objects, the robot was able to recognize, grasp and place the objects into appropriate class bins in 117 out of 135 trials.…”

Section: Well-labelled Datamentioning

confidence: 99%

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

2020

View full text Add to dashboard Cite

The motivation behind our work is to review and analyze the most relevant studies on deep reinforcement learning-based object manipulation. Various studies are examined through a survey of existing literature and investigation of various aspects, namely, the intended applications, techniques applied, challenges faced by researchers and recommendations for minimizing obstacles. This review refers to all relevant articles on deep reinforcement learning-based object manipulation and solutions. The object grasping issue is a major manipulation challenge. Object grasping requires detection systems, methods and tools to facilitate efficient and fast agent training. Several studies have proposed that object grasping and its subtypes are the main elements in dealing with the environment and agent. Unlike other review articles, this review article provides different observations on deep reinforcement learning-based manipulation. The results of this comprehensive review of deep reinforcement learning in the manipulation field may be valuable for researchers and practitioners because they can expedite the establishment of important guidelines.

show abstract

“…A grasping task requires critical empirical knowledge and skills from human beings. However, sensor noise and control imprecision continue to limit the robustness of robotic grasping in cluttered scenes [1]- [3].…”

Section: Introductionmentioning

confidence: 99%

“…1 TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, Universität Hamburg, 2 Agile Robots AG from knowledge of human nature [3], [7], [8], [15], [16]. However, high-precision empirical methods aimed at performing precise model-free grasping tasks are challenged by the limited number of objects and grasping labels in grasping databases.…”

Section: Introductionmentioning

confidence: 99%

Towards Precise Model-free Robotic Grasping with Sim-to-Real Transfer Learning

Zhang

Bai

Chen

et al. 2022

2022 IEEE International Conference on Robotics and Biomimetics (ROBIO)

View full text Add to dashboard Cite

Precise robotic grasping of several novel objects is a huge challenge in manufacturing, automation, and logistics. Most of the current methods for model-free grasping are disadvantaged by the sparse data in grasping datasets and by errors in sensor data and contact models. This study combines data generation and sim-to-real transfer learning in a grasping framework that reduces the sim-to-real gap and enables precise and reliable model-free grasping. A large-scale robotic grasping dataset with dense grasp labels is generated using domain randomization methods and a novel data augmentation method for deep learning-based robotic grasping to solve data sparse problem. We present an end-to-end robotic grasping network with a grasp optimizer. The grasp policies are trained with sim-to-real transfer learning. The presented results suggest that our grasping framework reduces the uncertainties in grasping datasets, sensor data, and contact models. In physical robotic experiments, our grasping framework grasped single known objects and novel complex-shaped household objects with a success rate of 90.91%. In a complex scenario with multi-objects robotic grasping, the success rate was 85.71%. The proposed grasping framework outperformed two state-of-the-art methods in both known and unknown object robotic grasping.

show abstract

Dex-Net MM: Deep Grasping for Surface Decluttering with a Low-Precision Mobile Manipulator

Cited by 5 publications

References 20 publications

CAD-based Grasp and Motion Planning for Process Automation in Fused Deposition Modelling

CAD-based Grasp and Motion Planning for Process Automation in Fused Deposition Modelling

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

Towards Precise Model-free Robotic Grasping with Sim-to-Real Transfer Learning

Contact Info

Product

Resources

About