Learning to Augment Synthetic Images for Sim2Real Policy Transfer

Abstract

Vision and learning have made significant progress that could improve robotics policies for complex tasks and environments. Learning deep neural networks for image understanding, however, requires large amounts of domain-specific visual data. While collecting such data from real robots is possible, such an approach limits the scalability as learning policies typically requires thousands of trials.In this work we attempt to learn manipulation policies in simulated environments. Simulators enable scalability and provide access to the underlying world state during training. Policies learned in simulators, however, do not transfer well to real scenes given the domain gap between real and synthetic data. We follow recent work on domain randomization and augment synthetic images with sequences of random transformations. Our main contribution is to optimize the augmentation strategy for sim2real transfer and to enable domain-independent policy learning. We design an efficient search for depth image augmentations using object localization as a proxy task. Given the resulting sequence of random transformations, we use it to augment synthetic depth images during policy learning. Our augmentation strategy is policy-independent and enables policy learning with no real images. We demonstrate our approach to significantly improve accuracy on three manipulation tasks evaluated on a real robot.

A Simple Model of Reading Eye Movement Based on Deep Learning

“…For image processing tasks, data augmentation is often performed through image rotation, reversing, and cropping [32]. However, these methods are not practical for text processing tasks.…”

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

“…The policy can also be transferred without a pre-trained dataset. Meanwhile, a cup placing policy was trained using CNN based on Monte Carlo tree search [167]. The purpose was to optimize the augmentation strategy for sim-to-real transfer and enable domain-independent policy learning.…”

Learning to combine primitive skills: A step towards versatile robotic manipulation §

“…Real robot transfer. To deploy our method on the real robot, we use a state-of-the-art technique of learning sim2real transfer based on data augmentation with domain randomization [14]. This method uses a proxy task of cube position prediction and a set of basic image transformations to learn a sim2real data augmentation function for depth images.…”

“…Our approach is shown to compare favorably to the state of the art [7] on the FetchPickPlace test environment [13]. Moreover, using recent techniques for domain adaptation [14] we demonstrate the successful transfer and high accuracy of our simulator-trained policies when tested on a real robot We compare our approach with two classical methods: (a) an open-loop controller estimating object positions and applying a standard motion planner (b) a closed-loop controller adapting the control to re-estimated object positions. We show the robustness of our approach to a variety of perturbations.…”