Accurate and Consistent Image-to-Image Conditional Adversarial Network

Abstract

Image-to-image translation based on deep learning has attracted interest in the robotics and vision community because of its potential impact on terrain analysis and image representation, interpretation, modification, and enhancement. Currently, the most successful approach for generating a translated image is a conditional generative adversarial network (cGAN) for training an autoencoder with skip connections. Despite its impressive performance, it has low accuracy and a lack of consistency; further, its training is imbalanced. This paper proposes a balanced training strategy for image-to-image translation, resulting in an accurate and consistent network. The proposed approach uses two generators and a single discriminator. The generators translate images from one domain to another. The discriminator takes the input of three different configurations and guides both the generators to generate realistic images in their corresponding domains while ensuring high accuracy and consistency. Experiments are conducted on different datasets. In particular, the proposed approach outperforms the cGAN in realistic image translation in terms of accuracy and consistency in training.

Depth Estimation From a Single RGB Image Using Fine-Tuned Generative Adversarial Network

“…The corresponding ground truth depth maps are shown in the second column. A comparative analysis is performed in terms of the absolute difference between the ground truth and generated depth maps for the proposed and CITN [28] approaches shown in the fourth column. The last column highlights the erroneous regions.…”

“…sufficient level of generality. We perform a comparative analysis of the proposed approach with the cGAN-based approach [31], BA-DualAE [29], CITN [28], MSDN [13], and FCN [36], respectively, in terms of three different datasets including the RealSense depth dataset [37], Cityscapes [15], and NYU dataset [16]. The comparative analysis for these three datasets using the above-mentioned approaches are given in the following Sections A, B, and C.…”

“…However, this approach requires the availability of the entire training set at the test time. Image-to-image translations [28][29][30][31][32], which translate images from one domain to another, play a key role in estimating depth from RGB images. Considering image-toimage translation, the authors in [29] recently proposed BA-DualAE, which is composed of two auto-encoders, where the latent spaces of the different domains are linked with a bidirectional regression network.…”

“…However, translating an RGB image to its corresponding depth map requires a careful selection of the target depth map for the given input RGB image, whereas such a condition becomes invalid when the corresponding objects in the target domain are not available [34]. Recently, a consistent imageto-image translation network (CITN) has been proposed in [28]. This network can translate images from one domain to another accurately, but at the cost of an increased number of parameters.…”

“…FIGURE 13. RGB input testing samples of the NYU dataset [16] in the first column translated into the depth maps shown in the third column by the proposed and CITN [28] approaches. The corresponding ground truth depth maps are shown in the second column.…”