Image-to-image translation based on deep learning has attracted interest in the robotics and vision community because of its potential impact on terrain analysis and image representation, interpretation, modification, and enhancement. Currently, the most successful approach for generating a translated image is a conditional generative adversarial network (cGAN) for training an autoencoder with skip connections. Despite its impressive performance, it has low accuracy and a lack of consistency; further, its training is imbalanced. This paper proposes a balanced training strategy for image-to-image translation, resulting in an accurate and consistent network. The proposed approach uses two generators and a single discriminator. The generators translate images from one domain to another. The discriminator takes the input of three different configurations and guides both the generators to generate realistic images in their corresponding domains while ensuring high accuracy and consistency. Experiments are conducted on different datasets. In particular, the proposed approach outperforms the cGAN in realistic image translation in terms of accuracy and consistency in training.
Automated image translation and completion is a subject of keen interest due to their impact on image representation, interpretation, and enhancement. To date, a conditional or a dual adversarial framework with a convolutional auto-encoder embedded as a generator is known to offer the best accuracy in image translation. However, although the frequency is excellent, the adversarial framework may suffer from a lack of generality, i.e., the accuracy drops when translating incomplete and corrupted data given as untrained noisy input data. This paper proposes an approach to robust image-to-image translation that offers a high level of generality while keeping accuracy high as well. The proposed approach is referred to here as a dual autoencoder with bidirectional latent space regression or Bi-directionally Associative DualAE, for short. The proposed BA-DualAE is configured with two auto-encoders the individual latent spaces of which are tightly associated by a bidirectional regression network. Once the two auto-encoders are trained independently for their respective domains, and then, the bidirectional regression network is trained to learn the general association between data pairs. With its capability of robust and bidirectional image translation, BA-DualAE performed direct image completion with no iterative search. The experiments with photo-sketch datasets demonstrated that the proposed BA-DualAE is highly robust under incomplete or corrupted data conditions and is far superior to adversarial frameworks in terms of generality and robustness.INDEX TERMS Convolutional neural network, auto-encoder, bidirectional latent space regression, imageto-image translation, generative adversarial network.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.