Attribute2Image: Conditional Image Generation from Visual Attributes

Yan, Xinchen; Yang, Jimei; Sohn, Kihyuk; Lee, Honglak

doi:10.48550/arxiv.1512.00570

Cited by 50 publications

(52 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After training, generation is done by sampling a vector from the latent space, concatenating it with the desired label and forwarding it through the decoder to obtain the output. More technical details regarding CVAEs can be found in [8], [31].…”

Section: Modeling and Conditional Training/generationmentioning

confidence: 99%

Dungeon and Platformer Level Blending and Generation using Conditional VAEs

Sarkar¹,

Cooper²

2021

Preprint

View full text Add to dashboard Cite

Variational autoencoders (VAEs) have been used in prior works for generating and blending levels from different games. To add controllability to these models, conditional VAEs (CVAEs) were recently shown capable of generating output that can be modified using labels specifying desired content, albeit working with segments of levels and platformers exclusively. We expand these works by using CVAEs for generating whole platformer and dungeon levels, and blending levels across these genres. We show that CVAEs can reliably control door placement in dungeons and progression direction in platformer levels. Thus, by using appropriate labels, our approach can generate whole dungeons and platformer levels of interconnected rooms and segments respectively as well as levels that blend dungeons and platformers. We demonstrate our approach using The Legend of Zelda, Metroid, Mega Man and Lode Runner.

show abstract

Section: Modeling and Conditional Training/generationmentioning

confidence: 99%

Dungeon and Platformer Level Blending and Generation using Conditional VAEs

Sarkar¹,

Cooper²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition to training a standard VAE on each sketch domain, we also trained a conditional VAE (CVAE) [27,37] on sketches from all domains taken together, with each sketch labeled with its corresponding domain. Conditional generative models [15], as the name suggests, enable generation of outputs conditioned on some given input.…”

Section: Conditional Sketch Generationmentioning

confidence: 99%

Multi-Domain Level Generation and Blending with Sketches via Example-Driven BSP and Variational Autoencoders

Snodgrass

Sarkar

2020

Preprint

View full text Add to dashboard Cite

Procedural content generation via machine learning (PCGML) has demonstrated its usefulness as a content and game creation approach, and has been shown to be able to support human creativity. An important facet of creativity is combinational creativity or the recombination, adaptation, and reuse of ideas and concepts between and across domains. In this paper, we present a PCGML approach for level generation that is able to recombine, adapt, and reuse structural patterns from several domains to approximate unseen domains. We extend prior work involving example-driven Binary Space Partitioning for recombining and reusing patterns in multiple domains, and incorporate Variational Autoencoders (VAEs) for generating unseen structures. We evaluate our approach by blending across 7 domains and subsets of those domains. We show that our approach is able to blend domains together while retaining structural components. Additionally, by using different groups of training domains our approach is able to generate both 1) levels that reproduce and capture features of a target domain, and 2) levels that have vastly different properties from the input domain.

show abstract

“…We showed its achievable via the proposed CoGAN framework. Note that our work is different to the Attribute2Image work [27], which is based on a conditional VAE model [28]. The conditional model can be used to generate images of different styles, but they are unsuitable for generating images in two different domains such as color and depth image domains.…”

Section: Related Workmentioning

confidence: 99%

Coupled Generative Adversarial Networks

Li¹,

Tuzel²

2016

Preprint

272

View full text Add to dashboard Cite

We propose coupled generative adversarial network (CoGAN) for learning a joint distribution of multi-domain images. In contrast to the existing approaches, which require tuples of corresponding images in different domains in the training set, CoGAN can learn a joint distribution without any tuple of corresponding images. It can learn a joint distribution with just samples drawn from the marginal distributions. This is achieved by enforcing a weight-sharing constraint that limits the network capacity and favors a joint distribution solution over a product of marginal distributions one. We apply CoGAN to several joint distribution learning tasks, including learning a joint distribution of color and depth images, and learning a joint distribution of face images with different attributes. For each task it successfully learns the joint distribution without any tuple of corresponding images. We also demonstrate its applications to domain adaptation and image transformation. To overcome the limitation, we propose the coupled generative adversarial networks (CoGAN) framework. It can learn a joint distribution of multi-domain images without existence of corresponding images in different domains in the training set. Only a set of images drawn separately from the marginal distributions of the individual domains is required. CoGAN is based on the generative adversarial networks (GAN) framework [5], which has been established as a viable solution for image distribution learning tasks. CoGAN extends GAN for joint image distribution learning tasks.CoGAN consists of a tuple of GANs, each for one image domain. When trained naively, the CoGAN learns a product of marginal distributions rather than a joint distribution. We show that by enforcing a weight-sharing constraint the CoGAN can learn a joint distribution without existence of corresponding images in different domains. The CoGAN framework is inspired by the idea that deep neural networks learn a hierarchical feature representation. By enforcing the layers that decode high-level semantics in the GANs to share the weights, it forces the GANs to decode the high-level semantics in the same way. The layers that decode low-level details then map the shared representation to images in individual domains for confusing the respective discriminative models. CoGAN is for multi-image domains but, for ease of presentation, we focused on the case of two image domains in the paper. However, the discussions and analyses can be easily generalized to multiple image domains.

show abstract

Attribute2Image: Conditional Image Generation from Visual Attributes

Cited by 50 publications

References 28 publications

Dungeon and Platformer Level Blending and Generation using Conditional VAEs

Dungeon and Platformer Level Blending and Generation using Conditional VAEs

Multi-Domain Level Generation and Blending with Sketches via Example-Driven BSP and Variational Autoencoders

Coupled Generative Adversarial Networks

Contact Info

Product

Resources

About