Benchmarking Structured Policies and Policy Optimization for Real-World Dexterous Object Manipulation

Funk, Niklas; Schaff, Charles; Madan, Rishabh; Yoneda, Takuma; Urain, Julen; Watson, Joe; Gordon, Ethan K.; Widmaier, Felix; Bauer, Stefan; Srinivasa, Siddhartha S.; Bhattacharjee, Tapomayukh; Walter, Matthew R.; Peters, Jan

doi:10.1109/lra.2021.3129139

Cited by 14 publications

(13 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluate our policy remotely on the TriFinger system [4] provided by the organisers of the real robot challenge [7]. The cube is tracked on the system using 3 cameras, described in [28].…”

Section: Policy Inference On Remote Real Robotmentioning

confidence: 99%

“…The aim in our 6-DoF manipulation task is to get the position and orientation of the cube to a specified goal position and orientation. We define our metric for 'success' in this task as getting the position within 2 cm, and orientation within 22°of the target goal pose as used in [3]; comparable to mean results obtained in [7]. Following previous works dealing with similar tasks [3,13,20], we attempted applying a reward based on the position and orientation components of error individually.…”

Section: Experiments 1: Trainingmentioning

confidence: 99%

“…Inference, including camera tracking and running the network, is performed on CPU on the same computer that hand-written solutions to last year's real robot challenge [7,8,9] were written on. An entire setup to run our system, including training, inference and physical robot hardware, could be purchased for less than US$10,000.…”

Section: A32 Hardware Setupmentioning

confidence: 99%

“…For the challenge the authors provided access to PyBullet simulation of the robot [6]. This challenge reinforced the inaccessibility of applying learning-based systems for such tasks: most competing teams elected to use structured policies with manually tuned high-level controllers or residual learning on top [7,8,9].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger

Allshire¹,

Mittal²,

Lodaya³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We present a system for learning a challenging dexterous manipulation task involving moving a cube to an arbitrary 6-DoF pose with only 3-fingers trained with NVIDIA's IsaacGym simulator. We show empirical benefits, both in simulation and sim-to-real transfer, of using keypoints as opposed to position+quaternion representations for the object pose in 6-DoF for policy observations and in reward calculation to train a model-free reinforcement learning agent. By utilizing domain randomization strategies along with the keypoint representation of the pose of the manipulated object, we achieve a high success rate of 83% on a remote TriFinger system maintained by the organizers of the Real Robot Challenge. With the aim of assisting further research in learning in-hand manipulation, we make the codebase of our system, along with trained checkpoints that come with billions of steps of experience available, at https://s2r2-ig.github.io

show abstract

Section: Policy Inference On Remote Real Robotmentioning

confidence: 99%

Section: Experiments 1: Trainingmentioning

confidence: 99%

Section: A32 Hardware Setupmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger

Allshire¹,

Mittal²,

Lodaya³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Deep reinforcement learning (RL) has witnessed remarkable progress over the last years, particularly in domains such as video games or other synthetic toy settings [1][2][3]. On the other hand, applying deep RL on real-world grounded robotic setup such as learning seemingly simple dexterous manipulation tasks in multi-object settings is still confronted with many fundamental limitations being the focus of many recent works [4][5][6][7][8][9][10][11]. The reinforcement learning problem in robotics setups is much more challenging [12].…”

Section: Introductionmentioning

confidence: 99%

Compositional Multi-Object Reinforcement Learning with Linear Relation Networks

Mambelli¹,

Träuble²,

Bauer³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Although reinforcement learning has seen remarkable progress over the last years, solving robust dexterous object-manipulation tasks in multi-object settings remains a challenge. In this paper, we focus on models that can learn manipulation tasks in fixed multi-object settings and extrapolate this skill zero-shot without any drop in performance when the number of objects changes. We consider the generic task of bringing a specific cube out of a set to a goal position. We find that previous approaches, which primarily leverage attention and graph neural network-based architectures, do not generalize their skills when the number of input objects changes while scaling as K 2 . We propose an alternative plug-and-play module based on relational inductive biases to overcome these limitations. Besides exceeding performances in their training environment, we show that our approach, which scales linearly in K, allows agents to extrapolate and generalize zero-shot to any new object number.

show abstract

Household Clothing Set and Benchmarks for Characterising End-Effector Cloth Manipulation

Clark,

Cramphorn-Neal,

Rachowiecki

et al. 2023

2023 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Benchmarking Structured Policies and Policy Optimization for Real-World Dexterous Object Manipulation

Cited by 14 publications

References 28 publications

Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger

Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger

Compositional Multi-Object Reinforcement Learning with Linear Relation Networks

Household Clothing Set and Benchmarks for Characterising End-Effector Cloth Manipulation

Contact Info

Product

Resources

About