Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models

Somepalli, Gowthami; Singla, Vasu; Goldblum, Micah; Geiping, Jonas; Goldstein, Tom

doi:10.48550/arxiv.2212.03860

Cited by 15 publications

(20 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Memorized images. We select eight image memorization examples from the recent works [64,10], four of which are shown in Figure 9. It also shows the sample generations before and after fine-tuning.…”

Section: Comparisons and Main Resultsmentioning

confidence: 99%

“…Current methods can synthesize high-quality images with remarkable generalization ability, capable of composing different instances, styles, and concepts in unseen contexts. However, as these models are often trained on copyright images, it learns to mimic various artist styles [64,61] and other copyrighted content [10].…”

Section: Related Workmentioning

confidence: 99%

“…Several works have studied training data leaking [62,12,13,11], which can pose a greater security and privacy risk, especially with the use of web-scale uncurated datasets in deep learning. Recent works [64,10] have also shown that text-to-image models are susceptible to generating exact or similar copies of the training dataset for certain text conditions. Another line of work in machine unlearning [9,21,23,22,42,8,66,60] explores data deletion at user's request after model training.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Ablating Concepts in Text-to-Image Diffusion Models

Kumari¹,

Zhang²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Section: Comparisons and Main Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Ablating Concepts in Text-to-Image Diffusion Models

Kumari¹,

Zhang²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

“…Specifically, they proposed a simple and efficient method for extracting verbatim sequences from a language model's training set using only black-box query access. Recently, in the vision domain, Somepalli et al [246] showed that the data replication problem existed in diffusion models, where the generated images are close to the training data in terms of semantic similarity. To disclose worse-case privacy risk, Carlini et al [247] further explored the privacy vulnerabilities of state-of-the-art diffusion models by leveraging a generate-and-filter pipeline to extract over a thousand training examples from the models.…”

Section: Privacymentioning

confidence: 99%

A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT

Cao¹,

Li²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recently, ChatGPT, along with DALL-E-2 [1] and Codex [2],has been gaining significant attention from society. As a result, many individuals have become interested in related resources and are seeking to uncover the background and secrets behind its impressive performance. In fact, ChatGPT and other Generative AI (GAI) techniques belong to the category of Artificial Intelligence Generated Content (AIGC), which involves the creation of digital content, such as images, music, and natural language, through AI models. The goal of AIGC is to make the content creation process more efficient and accessible, allowing for the production of high-quality content at a faster pace. AIGC is achieved by extracting and understanding intent information from instructions provided by human, and generating the content according to its knowledge and the intent information. In recent years, large-scale models have become increasingly important in AIGC as they provide better intent extraction and thus, improved generation results. With the growth of data and the size of the models, the distribution that the model can learn becomes more comprehensive and closer to reality, leading to more realistic and high-quality content generation. This survey provides a comprehensive review on the history of generative models, and basic components, recent advances in AIGC from unimodal interaction and multimodal interaction. From the perspective of unimodality, we introduce the generation tasks and relative models of text and image. From the perspective of multimodality, we introduce the cross-application between the modalities mentioned above. Finally, we discuss the existing open problems and future challenges in AIGC.

show abstract

“…For example, in social networks, participation is usually public; recovering privately shared photos or messages from a model trained on social network data is the privacy violation. These kinds of attacks are referred to as training data reconstruction attacks, and have been successfully demonstrated against a number of machine learning models including language models (Carlini et al, 2021;Mireshghallah et al, 2022), generative models (Somepalli et al, 2022), and image classifiers (Balle et al, 2022;Haim et al, 2022). Recent work (Bhowmick et al, 2018;Balle et al, 2022;Guo et al, 2022a;Stock et al, 2022) has begun to provide evidence that if one is willing to forgo protection against membership inference, then the regime that protects against training data reconstruction is far larger, as predicted by the intuitive reasoning that successful reconstruction requires a significant number of bits about an individual example to be leaked by the model.…”

Section: Introductionmentioning

confidence: 99%

Bounding Training Data Reconstruction in DP-SGD

Hayes¹,

Mahloujifar²,

Balle³

2023

Preprint

View full text Add to dashboard Cite

Differentially private training offers a protection which is usually interpreted as a guarantee against membership inference attacks. By proxy, this guarantee extends to other threats like reconstruction attacks attempting to extract complete training examples. Recent works provide evidence that if one does not need to protect against membership attacks but instead only wants to protect against training data reconstruction, then utility of private models can be improved because less noise is required to protect against these more ambitious attacks. We investigate this further in the context of DP-SGD, a standard algorithm for private deep learning, and provide an upper bound on the success of any reconstruction attack against DP-SGD together with an attack that empirically matches the predictions of our bound. Together, these two results open the door to fine-grained investigations on how to set the privacy parameters of DP-SGD in practice to protect against reconstruction attacks. Finally, we use our methods to demonstrate that different settings of the DP-SGD parameters leading to the same DP guarantees can result in significantly different success rates for reconstruction, indicating that the DP guarantee alone might not be a good proxy for controlling the protection against reconstruction attacks.

show abstract

Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models

Cited by 15 publications

References 0 publications

Ablating Concepts in Text-to-Image Diffusion Models

Ablating Concepts in Text-to-Image Diffusion Models

A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT

Bounding Training Data Reconstruction in DP-SGD

Contact Info

Product

Resources

About