Implicit sampling for hierarchical Bayesian inversion and applications in fractional multiscale diffusion models

Song, Xiaoyan; Jiang, Lijian; Zheng, Guang-Hui

doi:10.1016/j.cam.2020.112826

Cited by 45 publications

(105 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…And as shown in the 'Denoising' row of Figure 3, with strong noise injected to the input, the model is able to recover objects that are almost invisible. This finding potentially connects our pretraining method with genitive models (Song et al, 2021a;Ho et al, 2020;Song & Ermon, 2019;Song et al, 2021b). We will further investigate the connections in future efforts.…”

Section: Self Comparisonsmentioning

confidence: 57%

“…In the s-time super-resolution (denoted as SR s×), the image are first downsampled using bicubic interpolation for s times, and resized back to the original size using nearest-neighbor interpolation. In the denoising experiments, we take a noise scheme inspired by diffusion models (Song et al, 2021a;Ho et al, 2020) with ↓ (x) = √ γx+ √ 1 − γ , with ∼ N (0, I) and γ uniformly sampled as γ ∼ U(0, 1).…”

Section: Self Comparisonsmentioning

confidence: 99%

See 1 more Smart Citation

Energy-Inspired Self-Supervised Pretraining for Vision Models

Wang¹,

Wang²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Motivated by the fact that forward and backward passes of a deep network naturally form symmetric mappings between input and output representations, we introduce a simple yet effective self-supervised vision model pretraining framework inspired by energy-based models (EBMs). In the proposed framework, we model energy estimation and data restoration as the forward and backward passes of a single network without any auxiliary components, e.g., an extra decoder. For the forward pass, we fit a network to an energy function that assigns low energy scores to samples that belong to an unlabeled dataset, and high energy otherwise. For the backward pass, we restore data from corrupted versions iteratively using gradientbased optimization along the direction of energy minimization. In this way, we naturally fold the encoder-decoder architecture widely used in masked image modeling into the forward and backward passes of a single vision model. Thus, our framework now accepts a wide range of pretext tasks with different data corruption methods, and permits models to be pretrained from masked image modeling, patch sorting, and image restoration, including super-resolution, denoising, and colorization. We support our findings with extensive experiments, and show the proposed method delivers comparable and even better performance with remarkably fewer epochs of training compared to the state-of-the-art self-supervised vision model pretraining methods. Our findings shed light on further exploring self-supervised vision model pretraining and pretext tasks beyond masked image modeling.

show abstract

Section: Self Comparisonsmentioning

confidence: 57%

Section: Self Comparisonsmentioning

confidence: 99%

Energy-Inspired Self-Supervised Pretraining for Vision Models

Wang¹,

Wang²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…where x t = a t x 0 + σ t , and ω is a guidance weight. Sampling algorithms such as DDIM (Song et al, 2020a) and DPM-Solver (Lu et al, 2022a;Bao et al, 2022) are often adopted to speed up the sampling process of diffusion models. DDIM can also be utilized to deterministically reverse a sample x 0 back to its pure noise latent x T , enabling various image editing operations.…”

Section: Diffusion Modelsmentioning

confidence: 99%

Composer: Creative and Controllable Image Synthesis with Composable Conditions

Huang¹,

Chen²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recent large-scale generative models learned on big data are capable of synthesizing incredible images yet suffer from limited controllability. This work offers a new generation paradigm that allows flexible control of the output image, such as spatial layout and palette, while maintaining the synthesis quality and model creativity. With compositionality as the core idea, we first decompose an image into representative factors, and then train a diffusion model with all these factors as the conditions to recompose the input. At the inference stage, the rich intermediate representations work as composable elements, leading to a huge design space (i.e., exponentially proportional to the number of decomposed factors) for customizable content creation. It is noteworthy that our approach, which we call Composer, supports various levels of conditions, such as text description as the global information, depth map and sketch as the local guidance, color histogram for low-level details, etc. Besides improving controllability, we confirm that Composer serves as a general framework and facilitates a wide range of classical generative tasks without retraining. Code and models will be made available.

show abstract

“…There are three main hyperparameters that control the generation process. DDIM Steps controls the number of steps taken by the Denoising Diffusion Implicit Model [21] in the denoising process. More steps generally result in more realistic and coherent images, while fewer result in more disjointed surreal images.…”

Section: Text-to-imagementioning

confidence: 99%

Boosting Zero-shot Classification with Synthetic Data Diversity via Stable Diffusion

Shipard¹,

Wiliem²,

Nguyen³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recent research has shown it is possible to perform zero-shot classification tasks by training a classifier with synthetic data generated by a diffusion model. However, the performance of this approach is still inferior to that of recent vision-language models. It has been suggested that the reason for this is a domain gap between the synthetic and real data. In our work, we show that this domain gap is not the main issue, and that diversity in the synthetic dataset is more important. We propose a bag of tricks to improve diversity and are able to achieve performance on par with one of the vision-language models, CLIP. More importantly, this insight allows us to endow zeroshot classification capabilities on any classification model.

show abstract

Implicit sampling for hierarchical Bayesian inversion and applications in fractional multiscale diffusion models

Cited by 45 publications

References 44 publications

Energy-Inspired Self-Supervised Pretraining for Vision Models

Energy-Inspired Self-Supervised Pretraining for Vision Models

Composer: Creative and Controllable Image Synthesis with Composable Conditions

Boosting Zero-shot Classification with Synthetic Data Diversity via Stable Diffusion

Contact Info

Product

Resources

About