Entropy-Driven Sampling and Training Scheme for Conditional Diffusion Generation

Zheng, Guangcong; Li, Shengming; Wang, Hui; Yao, Taiping; Chen, Yang; Ding, Shouhong; Li, Xi

doi:10.1007/978-3-031-20047-2_43

Cited by 7 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Diffusion model (Sohl-Dickstein et al 2015;Dhariwal and Nichol 2021) is a generative probability model, which has attracted many researchers' attention because of its highquality generative results. Diffusion models can successfully perform conditional image generation when trained with guidance such as semantic layout or class labels (Zheng et al 2022;Ramesh et al 2021;Saharia et al 2022b;Ho and Salimans 2022;Zheng et al 2023;Xue et al 2023).…”

Section: Related Work Conditional Diffusion Probabilistic Modelmentioning

confidence: 99%

SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model

Wu,

Li,

et al. 2024

AAAI

View full text Add to dashboard Cite

Controllable spherical panoramic image generation holds substantial applicative potential across a variety of domains. However, it remains a challenging task due to the inherent spherical distortion and geometry characteristics, resulting in low-quality content generation. In this paper, we introduce a novel framework of SphereDiffusion to address these unique challenges, for better generating high-quality and precisely controllable spherical panoramic images. For the spherical distortion characteristic, we embed the semantics of the distorted object with text encoding, then explicitly construct the relationship with text-object correspondence to better use the pre-trained knowledge of the planar images. Meanwhile, we employ a deformable technique to mitigate the semantic deviation in latent space caused by spherical distortion. For the spherical geometry characteristic, in virtue of spherical rotation invariance, we improve the data diversity and optimization objectives in the training process, enabling the model to better learn the spherical geometry characteristic. Furthermore, we enhance the denoising process of the diffusion model, enabling it to effectively use the learned geometric characteristic to ensure the boundary continuity of the generated images. With these specific techniques, experiments on Structured3D dataset show that SphereDiffusion significantly improves the quality of controllable spherical image generation and relatively reduces around 35% FID on average.

show abstract

Section: Related Work Conditional Diffusion Probabilistic Modelmentioning

confidence: 99%

SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model

Wu,

Li,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Diffusion models, a class of generative models based on deep learning [51] , [52] , [53] , [54] , have exhibited superior performance, generating highly realistic data across various domains. Notable applications are found in image generation [2] , [41] , [55] , [56] , [57] , [58] , [59] , [60] , image inpainting [61] , [62] , speech synthesis [63] , natural language processing [64] , [65] , [66] , [67] , [68] , temporal data modelling [69] , [70] , [71] , [72] , [73] , and multimodal modelling [39] , [41] , [55] , [74] . Diffusion-based generative models offer distinct advantages over other generative approaches, such as autoregressive models [75] , normalizing flows [76] , energy-based models [77] , variational auto-encoders (VAEs) [78] , and generative adversarial networks (GANs) [79] .…”

Section: Introductionmentioning

confidence: 99%