A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

Abstract

Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Neural Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets.In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

Deep learning in fault detection and diagnosis of building HVAC systems: A systematic review with meta analysis

“…On the contrary, training a VAE is relatively easy as gradient descent algorithms can be directly applied to minimize the reconstruction loss and the Kulback-Leibler divergence loss [101]. Although from the literature, data generated by a VAE are less variant, especially for high dimensional data such as text and images [102]. This is not a problem for the data of HVAC systems in general.…”

Games of GANs: game-theoretical models for generative adversarial networks

“…In contrast, the surveys in the second group ( [5, 8, 28-30]) focus on a specific issue in GANs (e.g., regularization methods, loss functions, etc) and address how researchers deal with such an issue. In the third category ( [7,9,19,[31][32][33][34][35][36][37][38][39][40][41][42][43]) the surveys summarize the application of GAN in a specific field, from computer vision and image synthesis to cybersecurity and anomaly detection. In the following, we briefly review surveys in each category and explain how our paper differs from them.…”

“…The authors in [7,9,[31][32][33][34][35][36][37][38] conducted reviews of different aspects of GAN progress in the field of computer vision and image synthesis. Cao et al [9] reviewed recently GAN models and their applications in computer vision.…”

“…In [35], Agnese et al proposed a taxonomy to summarize GAN-based text-to-image synthesis papers into four major categories: Semantic Enhancement GANs, Resolution Enhancement GANs, Diversity Enhancement GANS, and Motion Enhancement GANs. Different from the other surveys in this field, Sampath et al [37] examined the most recent developments of GANs techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are covered in this survey.…”

Deep learning-based neuron segmentation in quantitative phase images trained on procedurally generated data

“…On the opposite end of the field, generative adversarial networks (GANs) can be used to output highly realistic novel synthetic data points rather than abstract ones, which offers an alternative strategy for data oversampling. Interestingly, GANs can improve third party classifier performance by a few percentage points or by very large margins depending on the data set of interest, and on the particular adversarial formulation [16].…”