Classifier-Free Diffusion Guidance

Ho, Jonathan; Salimans, Tim

doi:10.48550/arxiv.2207.12598

Cited by 121 publications

(162 citation statements)

References 5 publications

Supporting

Mentioning

153

Contrasting

Order By: Relevance

“…Models such as Imagen (Saharia et al, 2022b) and DALL-E (Ramesh et al, 2022) are usually two-staged relying on the pre-trained models, requiring the alignment between the embedding vectors from two sources. GLIDE (Nichol et al, 2022) explores diffusion model with classifier-free (Ho & Salimans, 2022) guidance by setting guidance scale during training. The target space of these models is not discrete text space but stable vectors of pixel values.…”

Section: Related Workmentioning

confidence: 99%

DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

Shansan¹,

Mukai²,

Feng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusion models to natural language is difficult due to the discrete nature of text. We tackle this challenge by proposing DIFFUSEQ: a diffusion model designed for sequence-to-sequence (SEQ2SEQ) text generation tasks. Upon extensive evaluation over a wide range of SEQ2SEQ tasks, we find DIF-FUSEQ achieving comparable or even better performance than six established baselines, including a state-of-the-art model that is based on pre-trained language models. Apart from quality, an intriguing property of DIFFUSEQ is its high diversity during generation, which is desired in many SEQ2SEQ tasks. We further include a theoretical analysis revealing the connection between DIFFUSEQ and autoregressive/non-autoregressive models. Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks. 1

show abstract

Section: Related Workmentioning

confidence: 99%

DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

Shansan¹,

Mukai²,

Feng³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Classifier-free guidance [Ho and Salimans, 2022] is a widely used technique to improve sample quality while reducing diversity in conditional diffusion models, which jointly trains a single diffusion model on conditional and unconditional objectives via randomly dropping c during training (e.g. with 10% probability).…”

Section: Text-conditional Diffusion Modelmentioning

confidence: 99%

UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance

Li¹,

Xue²,

Xiao³

et al. 2022

Preprint

View full text Add to dashboard Cite

An epic dreamscape with fantasy architecture, vivid colors, wide angle, super highly detailed, professional digital painting Legendary elegant gnome hold map and feel confuse in forest, highly detailed, global illumination, ray tracing, sharp focus Beautiful village around an ancient dragon head, massive scale, realistic concept art, cinematic color scheme, dramatic lighting (a) Samples of complex scene image generation (b) Samples of simple scene image generation Beautiful robot female with closed eyes, sci-fi, fantasy, mythology, complex, elegant, highly detailed, digital painting A portrait of a pink cat with human blue eyes wearing a scarf and a top hat, high quality, painting Full body portrait of a squirrel girl, blush, cute and elegant, with squirrel tail cocked to the left

show abstract

“…Controllable diffusion models have been explored with classifier [9], classifier-free [28], and reconstruction [29] guidance for image and video generation. Li et al [14] use a diffusion model with different pre-trained classifiers to guide language generation on different natural language tasks.…”

Section: B Diffusion Modelingmentioning

confidence: 99%

Guided Conditional Diffusion for Controllable Traffic Simulation

Zhong¹,

Rempe²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles. Typical heuristic-based traffic models offer flexible control to make vehicles follow specific trajectories and traffic rules. On the other hand, data-driven approaches generate realistic and human-like behaviors, improving transfer from simulated to real-world traffic. However, to the best of our knowledge, no traffic model offers both controllability and realism. In this work, we develop a conditional diffusion model for controllable traffic generation (CTG) that allows users to control desired properties of trajectories at test time (e.g., reach a goal or follow a speed limit) while maintaining realism and physical feasibility through enforced dynamics. The key technical idea is to leverage recent advances from diffusion modeling and differentiable logic to guide generated trajectories to meet rules defined using signal temporal logic (STL). We further extend guidance to multi-agent settings and enable interaction-based rules like collision avoidance. CTG is extensively evaluated on the nuScenes dataset for diverse and composite rules, demonstrating improvement over strong baselines in terms of the controllability-realism tradeoff.

show abstract

Classifier-Free Diffusion Guidance

Cited by 121 publications

References 5 publications

DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance

Guided Conditional Diffusion for Controllable Traffic Simulation

Contact Info

Product

Resources

About