Flow Contrastive Estimation of Energy-Based Models

Gao, Ruiqi; Nijkamp, Erik; Kingma, Diederik P.; Xu, Zhen; Dai, Andrew M.; Wu, Ying

doi:10.1109/cvpr42600.2020.00754

Cited by 53 publications

(78 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Energy Based Models. Our work is related to existing work on energy-based models [5,7,9,11,13,23,34,43,46]. Most similar to our work is that of [8], which proposes a framework of utilizing EBMs to compose several object descriptions together.…”

Section: Generated Imagementioning

confidence: 90%

Learning to Compose Visual Relations

Liu¹,

Li²,

Du³

et al. 2021

Preprint

View full text Add to dashboard Cite

The visual world around us can be described as a structured set of objects and their associated relations. An image of a room may be conjured given only the description of the underlying objects and their associated relations. While there has been significant work on designing deep neural networks which may compose individual objects together, less work has been done on composing the individual relations between objects. A principal difficulty is that while the placement of objects is mutually independent, their relations are entangled and dependent on each other. To circumvent this issue, existing works primarily compose relations by utilizing a holistic encoder, in the form of text or graphs. In this work, we instead propose to represent each relation as an unnormalized density (an energy-based model), enabling us to compose separate relations in a factorized manner. We show that such a factorized decomposition allows the model to both generate and edit scenes that have multiple sets of relations more faithfully. We further show that decomposition enables our model to effectively understand the underlying relational scene structure. Project page at: https://composevisualrelations.github.io/ * indicates equal contribution 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

show abstract

Section: Generated Imagementioning

confidence: 90%

Learning to Compose Visual Relations

Liu¹,

Li²,

Du³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Generative Adversarial Networks DCGAN [181] 37.11 -ProGAN [114] 15.52 -BigGAN [19] 14.73 -StyleGAN2 + ADA [115] 2.42 -Energy Based Models IGEBM [46] 37.9 -Denoising Diffusion [87] () 3.17 ≤ 3.75 DDPM++ Continuous [205] () 2.20 -Flow Contrastive (EBM) [55] 37.30 ≈ 3.27 VAEBM [246] 12.19 -…”

Section: Fid Nll (In Bpd)mentioning

confidence: 99%

“…where p θ (x) = e E θ (x)−c . This approach can be used to train a correction via exponential tilting [165], but can also be used to directly train an EBM and normalizing flow [55]. Eqn.…”

Section: Noise Contrastive Estimationmentioning

confidence: 99%

Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models

Bond-Taylor

Leach

Long

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

194

102

View full text Add to dashboard Cite

Deep generative models are a class of techniques that train deep neural networks to model the distribution of training samples. Research has fragmented into various interconnected approaches, each of which make trade-offs including run-time, diversity, and architectural restrictions. In particular, this compendium covers energy-based models, variational autoencoders, generative adversarial networks, autoregressive models, normalizing flows, in addition to numerous hybrid approaches. These techniques are compared and contrasted, explaining the premises behind each and how they are interrelated, while reviewing current state-of-the-art advances and implementations.

show abstract

“…Our work draws on recent work in energy based models (EBMs) [12,14,19,21,34,43,47,53]. Our underlying energy optimization procedure to generate samples is reminiscent of Langevin sampling, which is used to sample from EBMs [12,43,53].…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Learning of Compositional Energy Concepts

Du¹,

Li²,

Sharma³

et al. 2021

Preprint

View full text Add to dashboard Cite

Humans are able to rapidly understand scenes by utilizing concepts extracted from prior experience. Such concepts are diverse, and include global scene descriptors, such as the weather or lighting, as well as local scene descriptors, such as the color or size of a particular object. So far, unsupervised discovery of concepts has focused on either modeling the global scene-level or the local object-level factors of variation, but not both. In this work, we propose COMET, which discovers and represents concepts as separate energy functions, enabling us to represent both global concepts as well as objects under a unified framework. COMET discovers energy functions through recomposing the input image, which we find captures independent factors without additional supervision. Sample generation in COMET is formulated as an optimization process on underlying energy functions, enabling us to generate images with permuted and composed concepts. Finally, discovered visual concepts in COMET generalize well, enabling us to compose concepts between separate modalities of images as well as with other concepts discovered by a separate instance of COMET trained on a different dataset * .

show abstract

Flow Contrastive Estimation of Energy-Based Models

Cited by 53 publications

References 39 publications

Learning to Compose Visual Relations

Learning to Compose Visual Relations

Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models

Unsupervised Learning of Compositional Energy Concepts

Contact Info

Product

Resources

About