Occlusion-Aware GAN for Face De-Occlusion in the Wild

et al. 2020

Preprint

Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets. In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examine key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and further provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

Section: Imbalance Due To Occlusions In Segmentationmentioning

confidence: 99%

“…Occlusion-Aware GAN [230] Occlusion free image generation Table 3:Comparative summary of GANs for the problem of imbalances in computer vision…”

Section: Multi Class Classificationmentioning

confidence: 99%

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

et al. 2020

Preprint

“…The generator and discriminator are trained to generate an object image in which the invisible regions of the object are reconstructed. Dong et al [232] proposed a two stage model, named Occlusion-Aware GAN (OA-GAN), to remove arbitrary facial occlusions, e.g., faces with mask, microphone, cigarette, etc. OA-GAN is equipped with two GANs: The first GAN 1 is designed to disentangle the occlusion, and the second GAN 2 is trained to generate the occlusion free images given the generated occlusions.…”

Section: Imbalance Due To Occlusions In Segmentationmentioning

confidence: 99%

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

et al. 2020

Preprint

Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets.In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

“…SeGAN [226], developed by Ehsani et al, is an interesting framework to segment the invisible part of the object and then generate the appearance by painting the invisible parts. The proposed framework uses a segmentor, a generator, and a discriminator to combine segmentation and generation tasks ( Figure 33 Dong et al [227] proposed a two stage model, named Occlusion-Aware GAN (OA-GAN), to remove arbitrary facial occlusions, e.g., faces with mask, microphone, cigarette, etc. OA-GAN is equipped with two GANs: The first GAN is designed to disentangle the occlusion, and the second GAN is trained to generate the occlusion free images given the generated occlusions.…”

Section: Imbalance Due To Occlusions In Segmentationmentioning

confidence: 99%

“…Occlusion-Aware GAN [227] Occlusion free image generation Table 3:Comparative summary of GANs for the problem of imbalances in computer vision…”

Section: Multi Class Classificationmentioning

confidence: 99%

A Survey on Generative Adversarial Networks for Imbalance Problems in Computer Vision Tasks

et al. 2020

Preprint

Any computer vision application development starts off by acquiring images and data, then preprocessingand pattern recognition steps to perform a task. When the acquired image is highly imbalanced and notadequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems inacquired image datasets in certain complex real-world problems such as anomaly detection, emotionrecognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction,etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when thetraining dataset is imbalanced. In recent years, Generative Adversarial Networks (GANs) have gainedimmense attention by researchers across a variety of application domains due to their capability to modelcomplex real-world image data. It is particularly important that GANs can not only be used to generatesynthetic images, but also its fascinating adversarial learning idea showed good potential in restoringbalance in imbalanced datasets.In this paper, we examine the most recent developments of GANs based techniques for addressingimbalance problems in image data. The real-world challenges and implementations of synthetic imagegeneration based on GANs are extensively covered in this survey. Our survey first introduces variousimbalance problems in computer vision tasks and its existing solutions, and then examine key conceptssuch as deep generative image models and GANs. After that, we propose taxonomy to summarize GANsbased techniques for addressing imbalance problems in computer vision tasks into three major categories:Image level imbalances in classification, object level imbalances in object detection and pixel levelimbalances in segmentation tasks. We elaborate the imbalance problems of each group, and furtherprovide GANs based solutions in each group. Readers will understand how GANs based techniques canhandle the problem of imbalances and boost performance of the computer vision algorithms.