Generative Adversarial Networks in Computer Vision

Abstract

Generative adversarial networks (GANs) have been extensively studied in the past few years. Arguably their most significant impact has been in the area of computer vision where great advances have been made in challenges such as plausible image generation, image-to-image translation, facial attribute manipulation, and similar domains. Despite the significant successes achieved to date, applying GANs to real-world problems still poses significant challenges, three of which we focus on here. These are as follows: (1) the generation of high quality images, (2) diversity of image generation, and (3) stabilizing training. Focusing on the degree to which popular GAN technologies have made progress against these challenges, we provide a detailed review of the state-of-the-art in GAN-related research in the published scientific literature. We further structure this review through a convenient taxonomy we have adopted based on variations in GAN architectures and loss functions. While several reviews for GANs have been presented to date, none have considered the status of this field based on their progress toward addressing practical challenges relevant to computer vision. Accordingly, we review and critically discuss the most popular architecture-variant, and loss-variant GANs, for tackling these challenges. Our objective is to provide an overview as well as a critical analysis of the status of GAN research in terms of relevant progress toward critical computer vision application requirements. As we do this we also discuss the most compelling applications in computer vision in which GANs have demonstrated considerable success along with some suggestions for future research directions. Codes related to the GAN-variants studied in this work is summarized on https://github.com/sheqi/GAN_Review.

A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost

“…GAN consists of two deep networks, discriminator and generator, which train synchronously during the learning step. The discriminator is optimized for the sake of distinguishing between genuine and created photos, while the generator is trained to fool the discriminator from distinguishing between genuine and created photos [ 21 ].…”

Deep learning as a tool for ecology and evolution

“…As the importance of the spatial component is becoming increasingly highlighted in population genetics (Bradburd and Ralph, 2019), deep learning is also beginning to be used for predicting sample origins based on genetic variation (Battey et al, 2020a) and local-ancestry inference (Montserrat et al, 2020), which aims to identify populations from which a genetic locus descended. This application involves using generative adversarial networks (GANs; Box 2) (Wang et al, 2019b) to create artificial human genomic sequences of known ancestry (Montserrat et al, 2019;Yelmen et al, 2019).…”

The value of mass-digitised cultural heritage content in creative contexts

“…As a new media artist, Martin engages in novel digital methods, helping cultural institutions attract new audiences by visualising collections data. Through exploring the use of Generative Adversarial Networks (GANs) – a form of machine learning (see Wang et al., 2019), Disley created novel artworks based on the Library’s large-scale digitised collections. For example, hundreds of openly licensed images of the Tay and Forth Bridges (National Library of Scotland, n.d.a, n.d.b) are reimagined as ghostly creations (Figure 1).…”