EDICT: Exact Diffusion Inversion via Coupled Transformations

Geography of Technology Transfer in China

“…After determining how to minimize L BPR-Diff using the aforementioned upper bound and analyzing the gradient, we proceed to validate the rationality of L BPR-Diff . Here, we establish a connection with the recently prominent Direct Preference Optimization (DPO) (Rafailov et al, 2023;Wallace et al, 2024;Meng et al, 2024), which has been shown to effectively align human feedback with large language models. For further details on DPO, we refer readers to (Rafailov et al, 2023).…”

Embedding an Ethical Mind: Aligning Text-to-Image Synthesis via Lightweight Value Optimization

“…Most existing studies, e.g., LLaVA [44,45] and KOSMOS [30,53], only focus on instructiontuning and primarily aim to endows I2T models with capabilities of finishing arbitrary natural language specified tasks like VQA. Besides, [38] and [77] apply RLHF and DPO to T2I respectively to achieve better alignment with prompt semantic meanings, rather than human values/ethics. Largely distinct from aforementioned works, we pay attention to aligning T2I (instead of I2T) models with human values (rather than task instructions or semantic meanings), so as to adaptively reduce the produced diverse risks corresponding to given value principles (not only one specific issue like debiasing), paving the way for safe development of multimodal generative models.…”

Photo-realistic image synthesis from lines and appearance with modular modulation

“…These approaches can be broadly categorized into three main lines of work: 1) fine-tuning DMs on carefully curated image-prompt datasets (Dai et al, 2023;Podell et al, 2023); 2) maximizing explicit reward functions, either through multi-step diffusion generation outputs (Prabhudesai et al, 2023;Clark et al, 2023;Lee et al, 2023) or policy gradient-based reinforcement learning (RL) methods (Fan et al, 2024;Black et al, 2023;Ye et al, 2024). 3) implicit reward maximization, exemplified by Diffusion-DPO (Wallace et al, 2024) and Diffusion-KTO (Yang et al, 2024), directly utilizes raw preference data without the need for explicit reward functions.…”