Learning dexterous in-hand manipulation

Abstract

We use reinforcement learning (RL) to learn dexterous in-hand manipulation policies that can perform vision-based object reorientation on a physical Shadow Dexterous Hand. The training is performed in a simulated environment in which we randomize many of the physical properties of the system such as friction coefficients and an object’s appearance. Our policies transfer to the physical robot despite being trained entirely in simulation. Our method does not rely on any human demonstrations, but many behaviors found in human manipulation emerge naturally, including finger gaiting, multi-finger coordination, and the controlled use of gravity. Our results were obtained using the same distributed RL system that was used to train OpenAI Five. We also include a video of our results: https://youtu.be/jwSbzNHGflM .

Demonstration-Guided Deep Reinforcement Learning of Control Policies for Dexterous Human-Robot Interaction

“…Our results are shown only in simulation and with a static target hand as an initial step towards natural humanrobot hand interactions. To achieve this level of performance on a real robot, transfer learning methods, such as the one suggested in [4], could be applied. We show that our policy reacts well to small velocity disturbances.…”

“…Recently, model-free DRL has been applied to the control problem of humanoid hands [2], achieving impressive results in a simulated environment. Furthermore, [4] demonstrates the possibility of transferring a policy trained in simulation to the real Shadow Dexterous Hand.…”

“…Recently, model-free deep reinforcement learning (DRL) algorithms have been applied to the control of humanoid hands, albeit on relatively simple tasks such as grasping or door opening [2] and in simulation only. In [4], a controller trained in simulation has been transferred to a real humanoid hand. This opens up the door to learn policies for natural physical human-robot interactions.…”

A survey of deep learning techniques for autonomous driving

“…Over 1 the course of the last decade, deep learning and artificial intelligence (AI) became the main technologies behind many breakthroughs in computer vision (Krizhevsky, Sutskever, & Hinton, 2012), robotics (Andrychowicz et al, 2018), and natural language processing (NLP; Goldberg, 2017). They also have a major impact in the autonomous driving revolution seen today both in academia and industry.…”

Benchmarking In-Hand Manipulation

“…With multi-fingered dexterous hands, in-hand manipulation has been performed leveraging the redundancy in the fingers to move the object without completely releasing the grasp [2], [4], [11], [18]- [21]. For under-actuated hands, model based control has been successfully employed [22]- [25].…”