A Robot Cluster for Reproducible Research in Dexterous Manipulation

Bauer, Sebastian; Widmaier, Felix; Wüthrich, Manuel; Buchholz, Annika; Stark, Sebastian; Goyal, Anirudh; Steinbrenner, Thomas; Akpo, Joel; Joshi, Shruti; Berenz, Vincent; Agrawal, Vaibhav; Funk, Niklas; Urain, Julen; Peters, Jan; Watson, Joe; Chen, Claire; Srinivasan, K; Zhang, Ruoyu; Zhang, Jeffrey; Walter, Matthew R.; Madan, Rishabh; Schaff, Charles; Maeda, Takahiro; Yoneda, Takuma; Yarats, Denis; Allshire, Arthur; Gordon, Ethan K.; Bhattacharjee, Tapomayukh; Srinivasa, Siddhartha S.; Garg, Animesh; Sikchi, Harshit; Wang, Jilong; Yao, Qingfeng; Yang, Shuyu; McCarthy, R. L.; Sanchez, Francisco Roldan; Wang, Qiang; Bulens, David Cordova; McGuinness, Kevin; O’Connor, Noel E.; Redmond, Stephen J.; Schölkopf, Bernhard

doi:10.48550/arxiv.2109.10957

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The costly nature of real-robot experimentation has limited research related to robotic dexterous manipulation. In light of these issues, the Real Robot Challenge (RRC) (Bauer et al, 2021) aims to advance the state-of-the-art in robotic manipulation by providing participants with remote access to a TriFinger robotic platform (see Figure 1b) (Wüthrich et al, 2020), allowing for free and easy real-robot experimentation. To further support ease of experimentation, users are also provided with a simulation of this robotic system (see Figure 1a).…”

Section: Introductionmentioning

confidence: 99%

Dexterous robotic manipulation using deep reinforcement learning and knowledge transfer for complex sparse reward‐based tasks

et al. 2022

View full text Add to dashboard Cite

This paper describes a deep reinforcement learning (DRL) approach that won Phase 1 of the Real Robot Challenge (RRC) 2021, and then extends this method to a more difficult manipulation task. The RRC consisted of using a TriFinger robot to manipulate a cube along a specified positional trajectory, but with no requirement for the cube to have any specific orientation. We used a relatively simple reward function, a combination of a goal-based sparse reward and a distance reward, in conjunction with Hindsight Experience Replay (HER) to guide the learning of the DRL agent (Deep Deterministic Policy Gradient [DDPG]). Our approach allowed our agents to acquire dexterous robotic manipulation strategies in simulation. These strategies were then deployed on the real robot and outperformed all other competition submissions, including those using more traditional robotic control techniques, in the final evaluation stage of the RRC. Here we extend this method, by modifying the task of Phase 1 of the RRC to require the robot to maintain the cube in a particular orientation, while the cube is moved along the required positional trajectory. The requirement to also orient the cube makes the agent less able to learn the task through blind exploration due to increased problem complexity. To circumvent this issue, we make novel use of a Knowledge Transfer (KT) technique that allows the strategies learned by the agent in the original task (which was agnostic to cube orientation) to be transferred to this task (where orientation matters). KT allowed the agent to learn and perform the extended task in the simulator, which improved the average positional deviation from 0.134 to 0.02 m, and average orientation deviation from 142 to 76 during evaluation. This KT concept shows good generalization properties and could be applied to any actor-critic learning algorithm.

show abstract

Section: Introductionmentioning

confidence: 99%

Dexterous robotic manipulation using deep reinforcement learning and knowledge transfer for complex sparse reward‐based tasks

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The costly nature of real-robot experimentation has limited research related to robotic dexterous manipulation. In light of these issues, the Real Robot Challenge (RRC) 1 aims to advance the state-of-the-art in robotic manipulation by providing participants with remote access to a TriFinger robotic platform (see Figure 1 (b)) 2 , allowing for free and easy real-robot experimentation. To further support ease of experimentation, users are also provided with a simulation of this robotic system (see Figure 1 (a)).…”

Section: Introductionmentioning

confidence: 99%

“…To further support ease of experimentation, users are also provided with a simulation of this robotic system (see Figure 1 (a)). Full details can be found in the 'Protocol' section of the RRC website 1 .…”

Section: Introductionmentioning

confidence: 99%

Dexterous Robotic Manipulation using Deep Reinforcement Learning and Knowledge Transfer for Complex Sparse Reward-based Tasks

Wang¹,

Sanchez²,

McCarthy³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper describes a deep reinforcement learning (DRL) approach that won Phase 1 of the Real Robot Challenge (RRC) 2021, and then extends this method to a more difficult manipulation task. The RRC consisted of using a TriFinger robot to manipulate a cube along a specified positional trajectory, but with no requirement for the cube to have any specific orientation. We used a relatively simple reward function, a combination of goal-based sparse reward and distance reward, in conjunction with Hindsight Experience Replay (HER) to guide the learning of the DRL agent (Deep Deterministic Policy Gradient (DDPG)). Our approach allowed our agents to acquire dexterous robotic manipulation strategies in simulation. These strategies were then applied to the real robot and outperformed all other competition submissions, including those using more traditional robotic control techniques, in the final evaluation stage of the RRC. Here we extend this method, by modifying the task of Phase 1 of the RRC to require the robot to maintain the cube in a particular orientation, while the cube is moved along the required positional trajectory. The requirement to also orient the cube makes the agent unable to learn the task through blind exploration due to increased problem complexity. To circumvent this issue, we make novel use of a Knowledge Transfer (KT) technique that allows the strategies learned by the agent in the original task (which was agnostic to cube orientation) to be transferred to this task (where orientation matters). KT allowed the agent to learn and perform the extended task in the simulator, which improved the average positional deviation from 0.134 m to 0.02 m, and average orientation deviation from 142 • to 76 • during evaluation. This KT concept shows good generalisation properties and could be applied to any actor-critic learning algorithm.

show abstract

A Robot Cluster for Reproducible Research in Dexterous Manipulation

Cited by 2 publications

References 0 publications

Dexterous robotic manipulation using deep reinforcement learning and knowledge transfer for complex sparse reward‐based tasks

Dexterous robotic manipulation using deep reinforcement learning and knowledge transfer for complex sparse reward‐based tasks

Dexterous Robotic Manipulation using Deep Reinforcement Learning and Knowledge Transfer for Complex Sparse Reward-based Tasks

Contact Info

Product

Resources

About