CRPGAN: Learning image-to-image translation of two unpaired images by cross-attention mechanism and parallelization strategy

Abstract

Unsupervised image-to-image translation (UI2I) tasks aim to find a mapping between the source and the target domains from unpaired training data. Previous methods can not effectively capture the differences between the source and the target domain on different scales and often leads to poor quality of the generated images, noise, distortion, and other conditions that do not match human vision perception, and has high time complexity. To address this problem, we propose a multi-scale training structure and a progressive growth generator method to solve UI2I task. Our method refines the generated images from global structures to local details by adding new convolution blocks continuously and shares the network parameters in different scales and also in the same scale of network. Finally, we propose a new Cross-CBAM mechanism (CRCBAM), which uses a multi-layer spatial attention and channel attention cross structure to generate more refined style images. Experiments on our collected Opera Face, and other open datasets Summer↔Winter, Horse↔Zebra, Photo↔Van Gogh, show that the proposed algorithm is superior to other state-of-art algorithms.

Single-image translation based on improved spatial attention and hybrid convolution

“…In our study, we compare our approach with other models, including a series of advanced image translation models, including TuiGAN, 11 contrastive learning for unpaired image-to-image translation (SinCUT), 24 single image texture translation for data augmentation (SITTA), 25 learning image-to-image translation of two unpaired images by cross-attention mechanism and parallelization strategy (CRPGAN), 13 and photorealistic style transfer via wavelet transforms (WCT2) 26 . Our goal is to gain insight into the performance differences of these models in image translation tasks.…”

“…One of its advantages is that it is suitable for training on small data sets and can handle many different I2I translation tasks. Feng et al 13 proposed CPRGAN, which gradually refines the global structure of the generated image to local details by continuously adding convolution blocks. At the same time, a cross-attention mechanism is introduced to generate more refined style images.…”