Fidel: Reconstructing Private Training Samples from Weight Updates in Federated Learning

Enthoven, David; Al-Ars, Zaid

doi:10.48550/arxiv.2101.00159

Cited by 6 publications

(13 citation statements)

References 4 publications

(5 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The complexity of these architectures vary with respect to the number of layers (depth), the number of neurons in each layer (width), and the type of connections between neurons. Our study shows that researchers tend to use simple architectures to evaluate their attacks in 30 (62%) papers, e.g., 1-layer CNN [33] or 1-layer MLP [12]. Only in 18 (38%) papers, the authors considered complex state-of-the-art CNN models, such as VGG [120], ResNet [52], and DenseNet [57], the winners of the famous ImageNet Large Scale Visual Recognition Challenge (ILSVRC) [110].…”

Section: Fallacies In Evaluation Setupsmentioning

confidence: 99%

“…Several model inversion attacks reconstruct the training data by exploiting the shared gradients [33], [136], [141]. In particular, they exploit mathematical properties of gradients in specific model architectures to infer in-formation about the input data.…”

Section: Fallacies In Evaluation Setupsmentioning

confidence: 99%

“…In addition, several attacks are evaluated with limited or impractical setups. For instance, some attacks are evaluated using oversimplified datasets (e.g., [157]) or simplified NN models (e.g., [33]). This in turn affects the generalizability of the experiments, results, and conclusions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Federated Learning Attacks Revisited: A Critical Discussion of Gaps, Assumptions, and Evaluation Setups

Wainakh¹,

Zimmer²,

Subedi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning (FL) enables a set of entities to collaboratively train a machine learning model without sharing their sensitive data, thus, mitigating some privacy concerns. However, an increasing number of works in the literature propose attacks that can manipulate the model and disclose information about the training data in FL. As a result, there has been a growing belief in the research community that FL is highly vulnerable to a variety of severe attacks. Although these attacks do indeed highlight security and privacy risks in FL, some of them may not be as effective in production deployment because they are feasible only under special-sometimes impractical-assumptions. Furthermore, some attacks are evaluated under limited setups that may not match real-world scenarios. In this paper, we investigate this issue by conducting a systematic mapping study of attacks against FL, covering 48 relevant papers from 2016 to the third quarter of 2021. On the basis of this study, we provide a quantitative analysis of the proposed attacks and their evaluation settings. This analysis reveals several research gaps with regard to the type of target ML models and their architectures. Additionally, we highlight unrealistic assumptions in the problem settings of some attacks, related to the hyper-parameters of the ML model and data distribution among clients. Furthermore, we identify and discuss several fallacies in the evaluation of attacks, which open up questions on the generalizability of the conclusions. As a remedy, we propose a set of recommendations to avoid these fallacies and to promote adequate evaluations.

show abstract

Section: Fallacies In Evaluation Setupsmentioning

confidence: 99%

Section: Fallacies In Evaluation Setupsmentioning

confidence: 99%

See 1 more Smart Citation

Federated Learning Attacks Revisited: A Critical Discussion of Gaps, Assumptions, and Evaluation Setups

Wainakh¹,

Zimmer²,

Subedi³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…They also proposed a new initialization mechanism to speed up the attack convergence. Unlike previous approaches, Enthoven et al [10] introduced an analytical attack that exploits fully-connected layers to reconstruct the input data on the server side, and they extend this exploitation to CNNs. Recently, Zhu et al [40] proposed a recursive closed-form attack.…”

Section: Data Reconstructionmentioning

confidence: 99%

User-Level Label Leakage from Gradients in Federated Learning

Wainakh

Ventola

Müßig

et al. 2022

Proceedings on Privacy Enhancing Technologies

View full text Add to dashboard Cite

Federated learning enables multiple users to build a joint model by sharing their model updates (gradients), while their raw data remains local on their devices. In contrast to the common belief that this provides privacy benefits, we here add to the very recent results on privacy risks when sharing gradients. Specifically, we investigate Label Leakage from Gradients (LLG), a novel attack to extract the labels of the users’ training data from their shared gradients. The attack exploits the direction and magnitude of gradients to determine the presence or absence of any label. LLG is simple yet effective, capable of leaking potential sensitive information represented by labels, and scales well to arbitrary batch sizes and multiple classes. We mathematically and empirically demonstrate the validity of the attack under different settings. Moreover, empirical results show that LLG successfully extracts labels with high accuracy at the early stages of model training. We also discuss different defense mechanisms against such leakage. Our findings suggest that gradient compression is a practical technique to mitigate the attack.

show abstract

“…They also propose a new initialization mechanism to speed up the attack convergence. Unlike previous approaches, Enthoven et al [8] introduce an analytical attack that exploits fully-connected layers to reconstruct the input data on the server side, and they extend this exploitation to CNNs. Recently, Zhu et al [36] propose a recursive closed-form attack.…”

Section: Related Workmentioning

confidence: 99%

Label Leakage from Gradients in Distributed Machine Learning

Wainakh

Müßig

Grube

et al. 2021

2021 IEEE 18th Annual Consumer Communications &Amp; Networking Conference (CCNC)

View full text Add to dashboard Cite

Federated learning enables multiple users to build a joint model by sharing their model updates (gradients), while their raw data remains local on their devices. In contrast to the common belief that this provides privacy benefits, we here add to the very recent results on privacy risks when sharing gradients. Specifically, we propose Label Leakage from Gradients (LLG), a novel attack to extract the labels of the users' training data from their shared gradients. The attack exploits the direction and magnitude of gradients to determine the presence or absence of any label. LLG is simple yet effective, capable of leaking potential sensitive information represented by labels, and scales well to arbitrary batch sizes and multiple classes. We empirically and mathematically demonstrate the validity of our attack under different settings. Moreover, empirical results show that LLG successfully extracts labels with high accuracy at the early stages of model training. We also discuss different defense mechanisms against such leakage. Our findings suggest that gradient compression is a practical technique to prevent our attack.

show abstract

Fidel: Reconstructing Private Training Samples from Weight Updates in Federated Learning

Cited by 6 publications

References 4 publications

Federated Learning Attacks Revisited: A Critical Discussion of Gaps, Assumptions, and Evaluation Setups

Federated Learning Attacks Revisited: A Critical Discussion of Gaps, Assumptions, and Evaluation Setups

User-Level Label Leakage from Gradients in Federated Learning

Label Leakage from Gradients in Distributed Machine Learning

Contact Info

Product

Resources

About