Efficient Representations of Object Geometry for Reinforcement Learning of Interactive Grasping Policies

Mosbach, Malte; Behnke, Sven

doi:10.1109/irc55401.2022.00034

Cited by 5 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, researchers have investigated the use of reinforcement learning for multi-objective robotic tasks [14][15][16]. Agents can optimize several objectives by learning a set of trade-off policies using multi-objective reinforcement learning (MORL) algorithms.…”

Section: Related Workmentioning

confidence: 99%

Research on Perception and Control Technology for Dexterous Robot Operation

Zhang

2023

Electronics

View full text Add to dashboard Cite

Robotic grasping in cluttered environments is a fundamental and challenging task in robotics research. The ability to autonomously grasp objects in cluttered scenes is crucial for robots to perform complex tasks in real-world scenarios. Conventional grasping is based on the known object model in a structured environment, but the adaptability of unknown objects and complicated situations is constrained. In this paper, we present a robotic grasp architecture of attention-based deep reinforcement learning. To prevent the loss of local information, the prominent characteristics of input images are automatically extracted using a full convolutional network. In contrast to previous model-based and data-driven methods, the reward is remodeled in an effort to address the sparse rewards. The experimental results show that our method can double the learning speed in grasping a series of randomly placed objects. In real-word experiments, the grasping success rate of the robot platform reaches 90.4%, which outperforms several baselines.

show abstract

Section: Related Workmentioning

confidence: 99%

Research on Perception and Control Technology for Dexterous Robot Operation

Zhang

2023

Electronics

View full text Add to dashboard Cite

show abstract

“…In the first stage, we train the teacher policy to grasp objects represented by their oriented 3D bounding boxes. This representation is low-dimensional to allow for efficient learning while conveying enough information for specialized, geometry-aware behaviors to emerge [24]. We subsequently learn a student policy from segmented point clouds of the object to be grasped.…”

mentioning

confidence: 99%

Learning Generalizable Tool Use with Non-Rigid Grasp-Pose Registration

Mosbach,

Behnke

2023

2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)

Self Cite

View full text Add to dashboard Cite

Interactive grasping from clutter, akin to human dexterity, is one of the longest-standing problems in robot learning. Challenges stem from the intricacies of visual perception, the demand for precise motor skills, and the complex interplay between the two. In this work, we present Teacher-Augmented Policy Gradient (TAPG), a novel two-stage learning framework that synergizes reinforcement learning and policy distillation.After training a teacher policy to master the motor control based on object pose information, TAPG facilitates guided, yet adaptive, learning of a sensorimotor policy, based on object segmentation. We zero-shot transfer from simulation to a real robot by using Segment Anything Model for promptable object segmentation. Our trained policies adeptly grasp a wide variety of objects from cluttered scenarios in simulation and the real world based on human-understandable prompts. Furthermore, we show robust zero-shot transfer to novel objects. Videos of our experiments are available at https://maltemosbach. github.io/grasp_anything.

show abstract

Deep Reinforcement Learning of Dexterous Pre-Grasp Manipulation for Human-Like Functional Categorical Grasping

Pavlichenko,

Behnke

2023

2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)

View full text Add to dashboard Cite

Efficient Representations of Object Geometry for Reinforcement Learning of Interactive Grasping Policies

Cited by 5 publications

References 17 publications

Research on Perception and Control Technology for Dexterous Robot Operation

Research on Perception and Control Technology for Dexterous Robot Operation

Learning Generalizable Tool Use with Non-Rigid Grasp-Pose Registration

Deep Reinforcement Learning of Dexterous Pre-Grasp Manipulation for Human-Like Functional Categorical Grasping

Contact Info

Product

Resources

About