Inverting Gradients -- How easy is it to break privacy in federated learning?

Geiping, Jonas; Bauermeister, Hartmut; Dröge, Hannah; Moeller, Michael

doi:10.48550/arxiv.2003.14053

Cited by 68 publications

(157 citation statements)

References 0 publications

Supporting

Mentioning

155

Contrasting

Unclassified

Order By: Relevance

“…However, the design of FL still needs the protection of parameters as well as investigations on the tradeoffs between the privacy-security-level and the system performance. The study [152] suggested that FL could expose intermediate results such as stochastic gradient descent, and the transmission of these gradients may actually leak private information when exposed together with a data structure. It is still possible for adversaries to reconstruct the raw data approximately, especially when the architecture and parameters are not completely protected.…”

Section: ) Privacy Protection For Sus In Ss Networkmentioning

confidence: 99%

When Machine Learning Meets Spectrum Sharing Security: Methodologies and Challenges

Wang¹,

Sun²,

Hu³

et al. 2022

Preprint

View full text Add to dashboard Cite

The exponential growth of internet connected systems has generated numerous challenges, such as spectrum shortage issues, which require efficient spectrum sharing (SS) solutions. Complicated and dynamic SS systems can be exposed to different potential security and privacy issues, requiring protection mechanisms to be adaptive, reliable, and scalable. Machine learning (ML) based methods have frequently been proposed to address those issues. In this article, we provide a comprehensive survey of the recent development of ML based SS methods, the most critical security issues, and corresponding defense mechanisms. In particular, we elaborate the state-of-the-art methodologies for improving the performance of SS communication systems for various vital aspects, including ML based cognitive radio networks (CRNs), ML based database assisted SS networks, ML based LTE-U networks, ML based ambient backscatter networks, and other ML based SS solutions. We also present security issues from the physical layer and corresponding defending strategies based on ML algorithms, including Primary User Emulation (PUE) attacks, Spectrum Sensing Data Falsification (SSDF) attacks, jamming attacks, eavesdropping attacks, and privacy issues. Finally, extensive discussions on open challenges for ML based SS are also given. This comprehensive review is intended to provide the foundation for and facilitate future studies on exploring the potential of emerging ML for coping with increasingly complex SS and their security problems.

show abstract

Section: ) Privacy Protection For Sus In Ss Networkmentioning

confidence: 99%

When Machine Learning Meets Spectrum Sharing Security: Methodologies and Challenges

Wang¹,

Sun²,

Hu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Some attacks even allow an attacker to infer when a property appears and disappears in the dataset during the training process [34]. Sample inference attacks [106], [107] try to extract both the training data and their labels when attackers obtain model updates during the training phase. Recent work first adopt generate a dummy sample, then gradually reduce the distance between dummy simple and the grand truth through optimization algorithm [26], [108].…”

Section: B Privacy Threatsmentioning

confidence: 99%

“…Zhu [26] (De)centralized CV Zhao [108] Centralized CV Geiping [106] Centralized CV Yin [141] Centralized CV Dang [142] Centralized CV Jin [143] Federated CV Fu [144] ---Federated CV He [145] ----Federated CV al. observe that the aggregated gradient of an embedding layer is sparse with respect to the training text.…”

Section: Samplementioning

confidence: 99%

See 1 more Smart Citation

Robust and Privacy-Preserving Collaborative Learning: A Comprehensive Survey

Guo¹,

Zhang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

With the rapid demand of data and computational resources in deep learning systems, a growing number of algorithms to utilize collaborative machine learning techniques, for example, federated learning, to train a shared deep model across multiple participants. It could effectively take advantage of resource of each participant and obtain a more powerful learning system. However, integrity and privacy threats in such systems have greatly obstructed the applications of collaborative learning. And a large amount of works have been proposed to maintain the model integrity and mitigate the privacy leakage of training data during the training phase for different collaborate learning systems. Compared with existing surveys that mainly focus on one specific collaborate learning system, this survey aims to provide a systematic and comprehensive review of security and privacy researches in collaborative learning. Our survey first provides the system overview of collaborative learning, followed by an brief introduction of integrity and privacy threats. In an organized way, we then detail the existing integrity and privacy attacks as well as their defenses. We also list some open problems in this area and opensource the related papers on GitHub: https://github.com/csl-cqu/awesome-secure-collebrativelearning-papers.

show abstract

“…To reconstruct the samples given to a model, Zhu et al [1] propose Deep Leakage from Gradients (DLG), or Gradients Matching (GM), a method that iteratively updates a dummy training sample to minimize the distance of its gradient to a target gradient sent from a client. This has been used to successfully reconstruct large images in a mini-batch [5,6,7]. The application of GM has also been demonstrated in other domains, such as language modeling [1] and automatic speech recognition [8].…”

Section: Introductionmentioning

confidence: 99%

Revealing and Protecting Labels in Distributed Training

Dang

Thakkar

Ramaswamy

et al. 2021

Preprint

View full text Add to dashboard Cite

Distributed learning paradigms such as federated learning often involve transmission of model updates, or gradients, over a network, thereby avoiding transmission of private data. However, it is possible for sensitive information about the training data to be revealed from such gradients. Prior works have demonstrated that labels can be revealed analytically from the last layer of certain models (e.g., ResNet), or they can be reconstructed jointly with model inputs by using Gradients Matching [1] with additional knowledge about the current state of the model. In this work, we propose a method to discover the set of labels of training samples from only the gradient of the last layer and the id to label mapping. Our method is applicable to a wide variety of model architectures across multiple domains. We demonstrate the effectiveness of our method for model training in two domains -image classification, and automatic speech recognition. Furthermore, we show that existing reconstruction techniques improve their efficacy when used in conjunction with our method. Conversely, we demonstrate that gradient quantization and sparsification can significantly reduce the success of the attack.

show abstract

Inverting Gradients -- How easy is it to break privacy in federated learning?

Cited by 68 publications

References 0 publications

When Machine Learning Meets Spectrum Sharing Security: Methodologies and Challenges

When Machine Learning Meets Spectrum Sharing Security: Methodologies and Challenges

Robust and Privacy-Preserving Collaborative Learning: A Comprehensive Survey

Revealing and Protecting Labels in Distributed Training

Contact Info

Product

Resources

About