ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models

Choi, Joon-Seok; Kim, Sung-Won; Jeong, Yonghyun; Gwon, Youngjune; Yoon, Sungroh

doi:10.48550/arxiv.2108.02938

Cited by 50 publications

(84 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, denoising diffusion models were used to solve inverse problems in both supervised (i.e., degradation model is known during training) (Saharia et al, 2021b;a;Chung et al, 2021;Whang et al, 2021) and unsupervised settings (Kadkhodaie & Simoncelli, 2021;Kawar et al, 2021a;Jalal et al, 2021b;Song et al, 2021b;c;Choi et al, 2021). Unlike previous approaches, diffusion-based methods can successfully recover images from measurements with significant noise.…”

Section: Related Workmentioning

confidence: 99%

“…Our method, motivated by variational inference, obtains problem-specific, non-equilibrium update rules that lead to high-quality solutions in much fewer iterations. ILVR (Choi et al, 2021) suggests a diffusion-based method that handles noiseless super-resolution, and can run in 250 steps. In Appendix G, we prove that when applied on the same underlying generative diffusion model, ILVR is a special case of DDRM.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Denoising Diffusion Restoration Models

Kawar¹,

Elad²,

Ermon³

et al. 2022

Preprint

View full text Add to dashboard Cite

Many interesting tasks in image restoration can be cast as linear inverse problems. A recent family of approaches for solving these problems uses stochastic algorithms that sample from the posterior distribution of natural images given the measurements. However, efficient solutions often require problem-specific supervised training to model the posterior, whereas unsupervised methods that are not problem-specific typically rely on inefficient iterative methods. This work addresses these issues by introducing Denoising Diffusion Restoration Models (DDRM), an efficient, unsupervised posterior sampling method. Motivated by variational inference, DDRM takes advantage of a pre-trained denoising diffusion generative model for solving any linear inverse problem. We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization under various amounts of measurement noise. DDRM outperforms the current leading unsupervised methods on the diverse ImageNet dataset in reconstruction quality, perceptual quality, and runtime, being 5× faster than the nearest competitor. DDRM also generalizes well for natural images out of the distribution of the observed ImageNet training set. Our code is available at this GitHub repository.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Denoising Diffusion Restoration Models

Kawar¹,

Elad²,

Ermon³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We show results under trajectories of different number of timesteps K. We select the minimum K such that analytic-DPM can outperform the baselines with full timesteps and underline the corresponding results. (Chen et al, 2020;Kong et al, 2020;Popov et al, 2021;Lam et al, 2021), controllable generation (Choi et al, 2021;Sinha et al, 2021), image super-resolution (Saharia et al, 2021;, image-to-image translation (Sasaki et al, 2021), shape generation (Zhou et al, 2021) and time series forecasting (Rasul et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

Bao¹,

Li²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Diffusion probabilistic models (DPMs) represent a class of powerful generative models. Despite their success, the inference of DPMs is expensive since it generally needs to iterate over thousands of timesteps. A key problem in the inference is to estimate the variance in each timestep of the reverse process. In this work, we present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms w.r.t. its score function. Building upon it, we propose Analytic-DPM, a training-free inference framework that estimates the analytic forms of the variance and KL divergence using the Monte Carlo method and a pretrained score-based model. Further, to correct the potential bias caused by the score-based model, we derive both lower and upper bounds of the optimal variance and clip the estimate for a better result. Empirically, our analytic-DPM improves the log-likelihood of various DPMs, produces high-quality samples, and meanwhile enjoys a 20× to 80× speed up.

show abstract

“…Conditional generative methods Since we are interested in image registration that is performed given the moving and reference images, here we review the existing conditional diffusion-based models for image generation [9,19,30,34,37,39]. To control the image generation by the reverse process, DDIM [37] proposes a deterministic non-Markovian generative process starting from an initial condition.…”

Section: Diffusion Probabilistic Modelmentioning

confidence: 99%

“…DDPM learns the Markov transformation from Gaussian noise distribution to data distribution and provides diverse samples through stochastic diffusion processes. To generate images with desired semantics, conditional denoising diffusion models have been also presented [9,34]. However, it is challenging to apply DDPM to image registration tasks, since the existing methods may generate samples with losing image identity, and a correct registration should be performed by the deformation field for the moving image rather than image generation.…”

Section: Introductionmentioning

confidence: 99%

DiffuseMorph: Unsupervised Deformable Image Registration Along Continuous Trajectory Using Diffusion Models

Kim¹,

Han²,

Ye³

2021

Preprint

View full text Add to dashboard Cite

show abstract

ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models

Abstract: generation tasks, such as generation from various downsampling factors, multi-domain image translation, paint-toimage, and editing with scribbles.

Cited by 50 publications

References 48 publications

Denoising Diffusion Restoration Models

Denoising Diffusion Restoration Models

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

DiffuseMorph: Unsupervised Deformable Image Registration Along Continuous Trajectory Using Diffusion Models

Contact Info

Product

Resources

About