Deep Generative Models: A Review

Abstract

Objectives: To provide insight into deep generative models and review the most prominent and efficient deep generative models, including Variational Auto-encoder (VAE) and Generative Adversarial Networks (GANs). Methods: We provide a comprehensive overview of VAEs and GANs along with their advantages and disadvantages. This paper also surveys the recently introduced Attention-based GANs and the most recently introduced Transformer based GANs. Findings: GANs have been intensively researched because of their significant advantages over VAE. Furthermore, GANs are powerful generative models that have been widely employed in a variety of fields. Though GANs have a number of advantages over VAEs, but, despite their immense popularity and success, training GANs is still difficult and has experienced a lot of setbacks. These failures include mode collapse, where the generator produces the same set of outputs for various inputs, ultimately resulting in the loss of diversity; non-convergence due to oscillatory and diverging behaviors of the generator and discriminator during the training phase; and vanishing or exploding gradients, where learning either ceases to occur or occurs very slowly. Recently, some attention-based GANs and Transformer-based GANs have also been proposed for high-fidelity image generation. Novelty: Unlike previous survey articles, which often focus on all DGMs and dive into their complicated aspects, this work focuses on the most prominent DGMs, VAEs, and GANs and provides a theoretical understanding of them. Furthermore, because GAN is now the most extensively used DGM being studied by the academic community, the literature on it needs to be explored more. Moreover, while numerous articles on GANs are available, none have analyzed the most recent attention-based GANs and Transformer-based GANs. So, in this study, we review the recently introduced attention-based GANs and Transformer-based GANs, the literature related to which has not been reviewed by any survey paper.

Machine Learning and Deep Learning in Synthetic Biology: Key Architectures, Applications, and Challenges

“…The competition between these two networks helps the model generate increasingly realistic data. 85 Variational Autoencoders (VAEs). VAEs are probabilistic generative models that learn a probabilistic mapping between the data space and a latent space.…”

“…The generator creates synthetic data, and the discriminator’s role is to distinguish between real and generated data. The competition between these two networks helps the model generate increasingly realistic data …”

Raman Spectroscopy and AI Applications in Cancer Grading: An Overview

“…Additionally, even if synthetic data cannot fully replicate realworld scenarios, several AI techniques showcase the potential to generate synthetic yet reliable data, potentially bridging the gap between data-intensive, high-performing analyses and the more practical but less generalized current state of Raman data analysis. To tackle these challenges, a variety of deep generative methods can be utilized, such as GANs, variational autoencoders, diffusion models, flow-based models, and energy-based models, leading to new possibilities within the existing resources [196]- [198].…”

Fintech's Generative AI Revolution How AI is shaping the Future of Banking and Financial Services

“…VAEs capture the underlying distribution of data and can generate new data samples based on that distribution. (Mehmood et al, 2023) Numerous studies and research papers have highlighted the benefits of using Generative AI in the financial services sector (Kini & Basri, 2022). Some of the key benefits identified include:…”