Adversarial Learning With Knowledge of Image Classification for Improving GANs

Abstract

Generating realistic images with fine details are still challenging due to difficulties of training GANs and mode collapse. To resolve this problem, our main idea is that leveraging the knowledge of an image classification network, which is pre-trained by a large scale dataset (e.g. ImageNet), would improve a GAN. By using the gradient of the network (i.e. discriminator) with high discriminability during training, we can, therefore, guide the gradient of a generator gradually toward the real data region. However, excessive negative feedback of the powerful classifier often prevents a generator from producing diverse images. Based on the main idea, we design a GAN including the added discriminator and propose a novel energy function in order to transfer the pre-trained knowledge to a generator and control the feedback of the added discriminator. We also present an incremental learning method to prevent the density of the generator to be the low-entropy distribution when training our GAN with respect to the energy function. We incorporate our method to DCGAN and demonstrate the ability to enhance the image quality even in high resolution on several datasets compared to DCGAN. In addition, we compare our method with recent GANs for the diversity of generated samples on CIFAR-10 and STL-10 datasets and provide the extensive ablation studies to prove the benefits of our method. INDEX TERMS Generative adversarial network, image classification network, image generation, generative model, deep learning, convolutional neural network.

Classification of pathogens by Raman spectroscopy combined with generative adversarial networks

“…The random forest and support vector machine algorithms in machine learning and the CNN algorithm in deep learning are all discriminative techniques, while GANs belong to the class of generative techniques. Unlike discriminative techniques, generative techniques do not require a large amount of training data (Baek et al, 2019). A GAN is composed of two neural networks, a generator G and a discriminator D, which compete with each other based on the available training data to improve their performance (Lin et al, 2017).…”

Vis–NIR Spectroscopy Combined with GAN Data Augmentation for Predicting Soil Nutrients in Degraded Alpine Meadows on the Qinghai–Tibet Plateau

“…GAN is a deep learning algorithm that was first proposed by Goodfellow et al [ 53 ], based on the idea of binary zero-sum games in game theory, which can be trained to learn the distribution pattern of data to generate fake data that is very close to the real data. Compared to traditional machine learning generation algorithms such as quadratic discriminant analysis (QDA) and K-nearest neighbor (KNN), GAN is a generation technique that does not require much training data [ 54 ]. The main structure of a GAN consists of two parts: the generator (G) and the discriminator (D).…”

TilGAN: GAN for Facilitating Tumor-Infiltrating Lymphocyte Pathology Image Synthesis With Improved Image Classification

“…For the quantitative evaluation of the images generated by our proposed TilGAN model, we used the Inception score (IS) [ 15 ], kernel Inception distance (KID) [ 62 ], and Fréchet Inception distance (FID) [ 63 ]. All the scores were calculated using a pretrained Inception-v3 network [ 59 ], [ 64 ]. We calculated the IS as follows [ 59 ], [ 65 ]: …”

“…All the scores were calculated using a pretrained Inception-v3 network [ 59 ], [ 64 ]. We calculated the IS as follows [ 59 ], [ 65 ]: …”