In2I: Unsupervised Multi-Image-to-Image Translation Using Generative Adversarial Networks

Perera, Pramuditha; Abavisani, Mahdi; Patel, Vishal M.

doi:10.1109/icpr.2018.8545464

Cited by 31 publications

(19 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, when we compare the two ICVL figures [ 11 , 12 ] with the two previous CAVE figures [ 8 , 9 ], respectively, we may notice that the identical method performs more effectively on the ICVL scenes than on the CAVE scenes. We think that the main reason for this is that since most of the CAVE images are taken indoors, only a few bright objects contain useful information, and the main dark background contains scarce data to train the model.…”

Section: Methodsmentioning

confidence: 99%

“…Moreover, most researchers believe that complex deep neural networks would improve the models’ performance, and the more complex the neural networks are, the better effect they will have. Therefore, their approaches have often adopted tens or hundreds of convolutional neural network (CNN) layers with hundreds of residual network layers, which leads to a long computing time and high power consumption [ 11 , 12 , 13 ]. According to our thorough investigations and experiments, normal 2D convolutional operations scarcely improve the spectrum reconstruction performance and make the generated images blurry.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

TaijiGNN: A New Cycle-Consistent Generative Neural Network for High-Quality Bidirectional Transformation between RGB and Multispectral Domains

Liu

Gherbi

et al. 2021

Sensors

View full text Add to dashboard Cite

Since multispectral images (MSIs) and RGB images (RGBs) have significantly different definitions and severely imbalanced information entropies, the spectrum transformation between them, especially reconstructing MSIs from RGBs, is a big challenge. We propose a new approach, the Taiji Generative Neural Network (TaijiGNN), to address the above-mentioned problems. TaijiGNN consists of two generators, G_MSI, and G_RGB. These two generators establish two cycles by connecting one generator’s output with the other’s input. One cycle translates the RGBs into the MSIs and converts the MSIs back to the RGBs. The other cycle does the reverse. The cycles can turn the problem of comparing two different domain images into comparing the same domain images. In the same domain, there are neither different domain definition problems nor severely underconstrained challenges, such as reconstructing MSIs from RGBs. Moreover, according to several investigations and validations, we effectively designed a multilayer perceptron neural network (MLP) to substitute the convolutional neural network (CNN) when implementing the generators to make them simple and high performance. Furthermore, we cut off the two traditional CycleGAN’s identity losses to fit the spectral image translation. We also added two consistent losses of comparing paired images to improve the two generators’ training effectiveness. In addition, during the training process, similar to the ancient Chinese philosophy Taiji’s polarity Yang and polarity Yin, the two generators update their neural network parameters by interacting with and complementing each other until they all converge and the system reaches a dynamic balance. Furthermore, several qualitative and quantitative experiments were conducted on the two classical datasets, CAVE and ICVL, to evaluate the performance of our proposed approach. Promising results were obtained with a well-designed simplistic MLP requiring a minimal amount of training data. Specifically, in the CAVE dataset, to achieve comparable state-of-the-art results, we only need half of the dataset for training; for the ICVL dataset, we used only one-fifth of the dataset to train the model, but obtained state-of-the-art results.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

TaijiGNN: A New Cycle-Consistent Generative Neural Network for High-Quality Bidirectional Transformation between RGB and Multispectral Domains

Liu

Gherbi

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The success of GANs in synthesizing realistic images has led researchers to explore the GAN framework for numerous applications such as data augmentation [41], zeroshot learning [81], image inpainting [71], image dehazing [67,75,69], text-to-image translation [66], image-to-image translation [22,78,43], texture synthesis [23], crowd-counting [55] and generating outdoor scenes from attributes [25]. Isola et al [22] proposed a general method for image-to-image translation using conditional adversar'ial networks.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

Riggan

et al. 2019

Int J Comput Vis

Self Cite

View full text Add to dashboard Cite

The large domain discrepancy between faces captured in polarimetric (or conventional) thermal and visible domain makes cross-domain face verification a highly challenging problem for human examiners as well as computer vision algorithms. Previous approaches utilize either a twostep procedure (visible feature estimation and visible image reconstruction) or an input-level fusion technique, where different Stokes images are concatenated and used as a multichannel input to synthesize the visible image given the corresponding polarimetric signatures. Although these methods have yielded improvements, we argue that input-level fusion alone may not be sufficient to realize the full potential of the available Stokes images. We propose a Generative Adversarial Networks (GAN) based multi-stream featurelevel fusion technique to synthesize high-quality visible images from prolarimetric thermal images. The proposed network consists of a generator sub-network, constructed using an encoder-decoder network based on dense residual

show abstract

“…Another issue with translation methods is that in the past, researchers have mainly focused on translation between two domains in the RGB modality. However, recently several researchers have had success with near-infrared (NIR) to RGB translation [29,28,23,18,22,27]. Also, CNN based translation methods help translate IR images to RGB for facial recognition [31,25].…”

Section: Image Translation Networkmentioning

confidence: 99%

Unsupervised object detection via LWIR/RGB translation

Abbott

Robertson

Rincón

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

In this work, we present two new methods to overcome the lack of annotated long-wavelength infrared (LWIR) data by exploiting the abundance of similar RGB imagery. We introduce a novel unsupervised adaptation to the cycleGAN architecture for translating non-corresponding LWIR/RGB datasets. Our ultimate goal is high detection rates in the synthetic RGB/real LWIR imagery using only RGB labelled imagery for training detection algorithms. We translate LWIR imagery to RGB, allowing us to use an RGB trained detection algorithm. We, thereby remove the need for labelled LWIR imagery for training detection algorithms. In addition, we translate RGB to LWIR to fine-tune a network for detection in real LWIR imagery. Experimental results show that our adaption helps to create synthetic RGB imagery with higher detection rates across two different datasets. We also find that combining the synthetic RGB and real LWIR imagery produces higher F1 scores on the RGB trained detection network. Fine-tuning detection networks with synthetic LWIR and testing with real LWIR imagery produce the highest F1 scores.

show abstract

In2I: Unsupervised Multi-Image-to-Image Translation Using Generative Adversarial Networks

Cited by 31 publications

References 19 publications

TaijiGNN: A New Cycle-Consistent Generative Neural Network for High-Quality Bidirectional Transformation between RGB and Multispectral Domains

TaijiGNN: A New Cycle-Consistent Generative Neural Network for High-Quality Bidirectional Transformation between RGB and Multispectral Domains

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

Unsupervised object detection via LWIR/RGB translation

Contact Info

Product

Resources

About