Applications and Limits of Image-to-Image Translation Models

Abstract

Image-to-image (121) translation models are widely employed in several fields, e.g., computer vision, security or medicine. Their goal is to map images from a source domain to a target domain while preserving content information.Despite their success, these models suffer from multiple weaknesses. For example, many practical scenarios d o not consent to collect a sufficient amount of images, leading to imbalanced domains. Furthermore, mode collapse and training instability require a careful design and further discourage their deployment o n edge devices. Finally, 121 models need an intensive computation to learn conditional probability distributions and are difficult to adapt to different contexts. These drawbacks mainly limit their large scale applicability.In this work, we want to shed light on the main solutions adopted to overcome the above issues and their impact on the performance.We also investigate several approaches to deploy these models on low-powered devices and weight sharing techniques to reduce the number of parameters and resources used. Index Terms-Image-to-image translation, GAN, cyclic loss I . INTRODUCTION This work was supported i n part b y the EC under project EdgeAI (101097300), b y the Italian MUR under PON project GLEAN, and b y project SERICS (PE00000014) under the MUR NRRP funded b y the EU -NextGenerationEU. Views and opinions expressed are however those o f the authors only and do not necessarily reflect those o f the European Union or the Italian MUR. Neither the European Union nor Italian MUR can be held responsible for them.