DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection

Rieger, Phillip; Nguyen, Thien Duc; Miettinen, Markus; Sadeghi, Ahmad-Reza

doi:10.14722/ndss.2022.23156

Cited by 44 publications

(44 citation statements)

References 6 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted that if the MA is less than 100%, misclassifications of the model can be counted in favor of the backdoor, especially if the model wrongly predicts the backdoor target. As already pointed out by Rieger et al [35], this phenomenon primarily occurs for image scenarios with pixel-based triggers. It causes the BA to be slightly higher than 0% for backdoor-free models.…”

Section: Overall Performancementioning

confidence: 55%

“…Image Classification (IC): We use the popular benchmark datasets MNIST, FMNIST, and CIFAR-10 in our experiments. As these datasets are frequently used for evaluating FL and backdoor attacks and defenses [3], [8], [14], [15], [16], [20], [23], [27], [30], [35], [42], [43], [44], [13], [34], it enables us to perform an equitable comparative analysis of our approach with other state-of-the-art approaches in the literature. All three consist of samples belonging to one out of ten classes, handwritten digits in the case of MNIST, articles of clothing in the case of FMNIST, and objects (airplanes, cars, birds, etc.)…”

Section: Methodsmentioning

confidence: 99%

“…For instance, some defenses are bypassed when multiple different backdoors are simultaneously inserted by different malicious clients [37]. Other defenses clip weights and add noise to negate the effect of malicious model updates, which reduces the benign performance of the global model [3], [26], [30], [40], or they make specific assumptions such as (i) the adversary inserts malicious updates (backdoors) in each training round [14], or (ii) the adversary attacks only at the end of the training [1], or (iii) the data of the benign clients having the same distribution [27], [30], [48], or (iv) each benign client must have a similar number of unique labels [35].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

BayBFed: Bayesian Backdoor Defense for Federated Learning

Kumari¹,

Rieger²,

Fereidooni³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is an emerging technology that allows participants to jointly train a machine learning model without sharing their private data with others. However, FL is vulnerable to poisoning attacks such as backdoor attacks. Consequently, a variety of defenses have recently been proposed, which have primarily utilized intermediary states of the global model (i.e., logits) or distance of the local models (i.e., L 2 − norm) with respect to the global model to detect malicious backdoors in FL. However, as these approaches directly operate on client updates (or weights), their effectiveness depends on factors such as clients' data distribution or the adversary's attack strategies. In this paper, we introduce a novel and more generic backdoor defense framework, called BayBFed, which proposes to utilize probability distributions over client updates to detect malicious updates in FL: BayBFed computes a probabilistic measure over the clients' updates to keep track of any adjustments made in the updates, and uses a novel detection algorithm that can leverage this probabilistic measure to efficiently detect and filter out malicious updates. Thus, it overcomes the shortcomings of previous approaches that arise due to the direct usage of client updates; nevertheless, our probabilistic measure will include all aspects of the local client training strategies. BayBFed utilizes two Bayesian Non-Parametric (BNP) extensions: (i) a Hierarchical Beta-Bernoulli process to draw a probabilistic measure given the clients' updates, and (ii) an adaptation of the Chinese Restaurant Process (CRP), referred by us as CRP-Jensen, which leverages this probabilistic measure to detect and filter out malicious updates. We extensively evaluate our defense approach on five benchmark datasets: CIFAR10, Reddit, IoT intrusion detection, MNIST, and FMNIST, and show that it can effectively detect and eliminate malicious updates in FL without deteriorating the benign performance of the global model. 1. In contrast, non-targeted poisoning attacks aim to deteriorate the performance of the global model on all test data points [7]. 2. As pointed out by Rieger et al., it is not realistic to assume validation data to be present on the aggregation server [35].

show abstract

Section: Overall Performancementioning

confidence: 55%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

BayBFed: Bayesian Backdoor Defense for Federated Learning

Kumari¹,

Rieger²,

Fereidooni³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Federated learning is susceptible to poisoning attacks by malicious participants, who may tamper with the local data or the local model parameters on the compromised clients. There has been substantial progress in recent years on modeling possible attacks and proposing defense methods against data poisoning [68] and model poisoning [22], [57], [62], [16]. Existing defenses can be adapted and integrated in CELEST to counter malicious interference during the federated learning training process.…”

Section: Discussion and Extensionsmentioning

confidence: 99%

CELEST: Federated Learning for Globally Coordinated Threat Detection

Ongun¹,

Boboila²,

Oprea³

et al. 2022

Preprint

View full text Add to dashboard Cite

The cyber-threat landscape has evolved tremendously in recent years, with new threat variants emerging daily, and large-scale coordinated campaigns becoming more prevalent. In this study, we propose CELEST (CollaborativE LEarning for Scalable Threat detection), a federated machine learning framework for global threat detection over HTTP, which is one of the most commonly used protocols for malware dissemination and communication. CELEST leverages federated learning in order to collaboratively train a global model across multiple clients who keep their data locally, thus providing increased privacy and confidentiality assurances. Through a novel active learning component integrated with the federated learning technique, our system continuously discovers and learns the behavior of new, evolving, and globally-coordinated cyber threats. We show that CELEST is able to expose attacks that are largely invisible to individual organizations. For instance, in one challenging attack scenario with data exfiltration malware, the global model achieves a three-fold increase in Precision-Recall AUC compared to the local model. We deploy CELEST on two university networks and show that it is able to detect the malicious HTTP communication with high precision and low false positive rates. Furthermore, during its deployment, CELEST detected a set of previously unknown 42 malicious URLs and 20 malicious domains in one day, which were confirmed to be malicious by VirusTotal.

show abstract

“…Existing defenses against such owners use Byzantine-robust aggregation rules such as trimmed mean [82], coordinate-wise mean [81] and Krum [11], which have been show to be susceptible to backdoor and model poisoning attacks [34]. Recent work in FL such as FLTrust [16] and DeepSight [71] provide mitigation against backdoor attacks. Both strategies are inherently heuristic, while SafeNet offers provable robustness guarantees.…”

Section: F Comparison With Federated Learningmentioning

confidence: 99%

SafeNet: Mitigating Data Poisoning Attacks on Private Machine Learning

Chaudhari¹,

Jagielski²,

Oprea³

2022

Preprint

View full text Add to dashboard Cite

Secure multiparty computation (MPC) has been proposed to allow multiple mutually distrustful data owners to jointly train machine learning (ML) models on their combined data. However, the datasets used for training ML models might be under the control of an adversary mounting a data poisoning attack, and MPC prevents inspecting training sets to detect poisoning. We show that multiple MPC frameworks for private ML training are susceptible to backdoor and targeted poisoning attacks. To mitigate this, we propose SafeNet, a framework for building ensemble models in MPC with formal guarantees of robustness to data poisoning attacks. We extend the security definition of private ML training to account for poisoning and prove that our SafeNet design satisfies the definition. We demonstrate SafeNet's efficiency, accuracy, and resilience to poisoning on several machine learning datasets and models. For instance, SafeNet reduces backdoor attack success from 100% to 0% for a neural network model, while achieving 39× faster training and 36× less communication than the four-party MPC framework of Dalskov et al. [26].

show abstract

DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection

Cited by 44 publications

References 6 publications

BayBFed: Bayesian Backdoor Defense for Federated Learning

BayBFed: Bayesian Backdoor Defense for Federated Learning

CELEST: Federated Learning for Globally Coordinated Threat Detection

SafeNet: Mitigating Data Poisoning Attacks on Private Machine Learning

Contact Info

Product

Resources

About