Closing the Reality Gap with Unsupervised Sim-to-Real Image Translation

Abstract

Deep learning approaches have become the standard solution to many problems in computer vision and robotics, but obtaining proper and sufficient training data is often a problem, as human labor is often error prone, time consuming and expensive. Solutions based on simulation have become more popular in recent years, but the gap between simulation and reality is still a major issue. In this paper, we introduce a novel model for augmenting synthetic image data through unsupervised image-to-image translation by applying the style of real world images to simulated images with open source frameworks. This model intends to generate the training data as a separate step and not as part of the training. The generated dataset is combined with conventional augmentation methods and is then applied to a neural network capable of running in real-time on autonomous soccer robots. Our evaluation shows a significant improvement compared to networks trained on simulated images without this kind of augmentation.