Adversarial discriminative sim-to-real transfer of visuo-motor policies

Abstract

Various approaches have been proposed to learn visuo-motor policies for real-world robotic applications. One solution is first learning in simulation then transferring to the real world. In the transfer, most existing approaches need real-world images with labels. However, the labelling process is often expensive or even impractical in many robotic applications. In this paper, we propose an adversarial discriminative sim-to-real transfer approach to reduce the cost of labelling real data. The effectiveness of the approach is demonstrated with modular networks in a table-top object reaching task where a 7 DoF arm is controlled in velocity mode to reach a blue cuboid in clutter through visual observations. The adversarial transfer approach reduced the labelled real data requirement by 50%. Policies can be transferred to real environments with only 93 labelled and 186 unlabelled real images. The transferred visuo-motor policies are robust to novel (not seen in training) objects in clutter and even a moving target, achieving a 97.8% success rate and 1.8 cm control accuracy.Some others tried to make use of both simulated and real data for a more balanced solution (Fitzgerald et al. 2015;

Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion

“…However, policies trained using simulations often do not transfer for real world tasks since the reality gap between simulations and the physical world are strongly pertinent. These issues of RL have been tackled using techniques such as actuator modeling [15], implementing a modular training approach [16], introducing domain randomization [17], increasing policy generalization [18] and adding noise to observations and actions in the training environments. Authors of [15] have demonstrated the use of a deep RL approach to complex legged locomotion tasks.…”

Quantifying the Reality Gap in Robotic Manipulation Tasks

“…This sim-to-real transfer problem has recently been tackled by numerous research groups. Earlier approaches mainly highlight the issues around this transfer [15], with more recent efforts proposing solutions, including domain adaptation and Generative Adversarial Networks (GAN) which requires both real world and simulated data [3]. Results showing the early promise of these techniques -using domain adaptation, Bousmalis et al [16] were able to achieve a success rate for real world grasping trained in simulation of 76.7% on a dataset of unseen objects.…”

A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies

“…This was recently applied to sim-to-real transfer for a robotic table-top-reaching task with a 7 DoF arm [8]. The authors show the ability to effectively transfer the learning of a visuomotor policies from a simulation environment to the real setup by the use of very few real expert demonstrations for fine-tuning.…”