Data Poisoning Attacks Against Federated Learning Systems

Abstract

Federated learning (FL) is an emerging paradigm for distributed training of large-scale deep neural networks in which participants' data remains on their own devices with only model updates being shared with a central server. However, the distributed nature of FL gives rise to new threats caused by potentially malicious participants. In this paper, we study targeted data poisoning attacks against FL systems in which a malicious subset of the participants aim to poison the global model by sending model updates derived from mislabeled data. We first demonstrate that such data poisoning attacks can cause substantial drops in classification accuracy and recall, even with a small percentage of malicious participants. We additionally show that the attacks can be targeted, i.e., they have a large negative impact only on classes that are under attack. We also study attack longevity in early/late round training, the impact of malicious participant availability, and the relationships between the two. Finally, we propose a defense strategy that can help identify malicious participants in FL to circumvent poisoning attacks, and demonstrate its effectiveness.

FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping

“…Data poisoning attacks have been demonstrated to many machine learning systems such as spam detection [32], [35], SVM [8], recommender systems [16], [17], [25], [45], neural networks [11], [18], [27], [30], [36], [37], and graph-based methods [20], [41], [49], as well as distributed privacy-preserving data analytics [10], [14]. FL is also vulnerable to data poisoning attacks [38], i.e., malicious clients can corrupt the global model via modifying, adding, and/or deleting examples in their local training datasets. For instance, a data poisoning attack known as label flipping attack changes the labels of the training examples on malicious clients while keeping their features unchanged.…”

Rebirth of Distributed AI—A Review of eHealth Research

“…Other researchers surveyed sub-problems of federated learning. For example, we find [ 18 ] focuses on personalization techniques in FL, while [ 19 ] focuses on data poisoning attacks against FL systems.…”

“…Although FL appears to ensure that data remain on-premises, recent studies have shown that there is still the possibility of an actor exploiting the shared updates to extract confidential data, maliciously influencing the model output, or causing other harm such as model malfunctioning. Based on the timing of the attack in respect to the model life cycle, major attacks can be categorized into: Attacks taking place during the model aggregation phase [ 19 , 95 , 96 ] Attacks taking place after the model is deployed [ 97 , 98 , 99 , 100 , 101 , 102 ] …”

“…Attacks taking place during the model aggregation phase [19,95,96] • Attacks taking place after the model is deployed [97][98][99][100][101][102] 7.6. Data Labeling Data labeling and pre-processing are important stages of machine learning.…”

Incentive-Driven Federated Learning and Associated Security Challenges: A Systematic Review

“…Data poisoning attacks can either be targeted or untargeted. Data poisoning attacks have been largely studied in the literature [93]- [96] and to counter such attacks, several defense strategies have been proposed in the literature [21]- [23]. These works proposed different aggregation rules that improve the robustness of the models against data poisoning attacks.…”