Efficient Detection of Byzantine Attacks in Federated Learning Using Last Layer Biases

Jebreel, Najeeb Moharram; Blanco-Justicia, Alberto; Sánchez, David; Domingo-Ferrer, Josep

doi:10.1007/978-3-030-57524-3_13

Cited by 8 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several works propose to analyze specific parts of the updates to counter poisoning attacks. [17] proposes analyzing the last layer's biases. However, it assumes the local data are iid and form two separate clusters.…”

Section: Related Workmentioning

confidence: 99%

FL-Defender: Combating Targeted Attacks in Federated Learning

Jebreel¹,

Domingo‐Ferrer²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Federated learning (FL) enables learning a global machine learning model from local data distributed among a set of participating workers. This makes it possible i) to train more accurate models due to learning from rich joint training data, and ii) to improve privacy by not sharing the workers' local private data with others. However, the distributed nature of FL makes it vulnerable to targeted poisoning attacks that negatively impact the integrity of the learned model while, unfortunately, being difficult to detect. Existing defenses against those attacks are limited by assumptions on the workers' data distribution, may degrade the global model performance on the main task and/or are ill-suited to high-dimensional models. In this paper, we analyze targeted attacks against FL and find that the neurons in the last layer of a deep learning (DL) model that are related to the attacks exhibit a different behavior from the unrelated neurons, making the last-layer gradients valuable features for attack detection. Accordingly, we propose FL-Defender as a method to combat FL targeted attacks. It consists of i) engineering more robust discriminative features by calculating the worker-wise angle similarity for the workers' last-layer gradients, ii) compressing the resulting similarity vectors using PCA to reduce redundant information, and iii) re-weighting the workers' updates based on their deviation from the centroid of the compressed similarity vectors. Experiments on three data sets with different DL model sizes and data distributions show the effectiveness of our method at defending against label-flipping and backdoor attacks. Compared to several state-of-the-art defenses, FL-Defender achieves the lowest attack success rates, maintains the performance of the global model on the main task and causes minimal computational overhead on the server.

show abstract

Section: Related Workmentioning

confidence: 99%

FL-Defender: Combating Targeted Attacks in Federated Learning

Jebreel¹,

Domingo‐Ferrer²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Wang et al [100] designed a secure federated learning system based on blockchain technology that can defend against Byzantine attacks. Jebreel et al [128] designed a novel concept against Byzantine attacks where the basic concept is the analysis of a small fraction of the updates, instead of analyzing the whole updates. Sun et al [45] proposed adaptive federated learning with digital twin, which is based on the concept of interaction records and learning quality that rely on the use of malicious updates to mitigate the malicious data threat.…”

Section: Byzantine Attackmentioning

confidence: 99%

Federated Deep Learning for Cyber Security in the Internet of Things: Concepts, Applications, and Experimental Analysis

Ferrag¹,

et al. 2021

View full text Add to dashboard Cite

In this article, we present a comprehensive study with an experimental analysis of federated deep learning approaches for cyber security in the Internet of Things (IoT) applications. Specifically, we first provide a review of the federated learning-based security and privacy systems for several types of IoT applications, including, Industrial IoT, Edge Computing, Internet of Drones, Internet of Healthcare Things, Internet of Vehicles, etc. Second, the use of federated learning with blockchain and malware/intrusion detection systems for IoT applications is discussed. Then, we review the vulnerabilities in federated learning-based security and privacy systems. Finally, we provide an experimental analysis of federated deep learning with three deep learning approaches, namely, Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), and Deep Neural Network (DNN). For each deep learning model, we study the performance of centralized and federated learning under three new real IoT traffic datasets, namely, the Bot-IoT dataset, the MQTTset dataset, and the TON_IoT dataset. The goal of this article is to provide important information on federated deep learning approaches with emerging technologies for cyber security. In addition, it demonstrates that federated deep learning approaches outperform the classic/centralized versions of machine learning (non-federated learning) in assuring the privacy of IoT device data and provide the higher accuracy in detecting attacks.

show abstract

“… Jebreel et al (2020) Game-theory approach The aggregation process is set up as a mixed-strategy game between the server and each client, with the server having the option to accept or reject each client's valid actions of sending good or bad model updatesChen et al (2022) Lim et al (2020) Residual-based Reweighting Iteratively Reweighted Least Squares (IRLS), which bases its reweighting technique on reweighting each parameter by its vertical distance (residual) from a robust regression line, is a method for enhancing the median-based aggregation operator Fu et al (2019) Adaptive Federated Averaging (AFA) AFA technology use the cosine similarity to measure the quality of model updates during training Reddi et al (2020) Muñoz-González et al (2019)…”

mentioning

confidence: 99%

Anomaly detection and defense techniques in federated learning: a comprehensive review

Zhang,

Yang,

Mao

et al. 2024

Artif Intell Rev

View full text Add to dashboard Cite

In recent years, deep learning methods based on a large amount of data have achieved substantial success in numerous fields. However, with increases in regulations for protecting private user data, access to such data has become restricted. To overcome this limitation, federated learning (FL) has been widely utilized for training deep learning models without centralizing data. However, the inaccessibility of FL data and heterogeneity of the client data render difficulty in providing security and protecting the privacy in FL. In addition, the security and privacy anomalies in the corresponding systems significantly hinder the application of FL. Numerous studies have been proposed aiming to maintain the model security and mitigate the leakage of private training data during the FL training phase. Existing surveys categorize FL attacks from a defensive standpoint, but lack the efficiency of pinpointing attack points and implementing timely defenses. In contrast, our survey comprehensively categorizes and summarizes detected anomalies across client, server, and communication perspectives, facilitating easier identification and timely defense measures. Our survey provides an overview of the FL system and briefly introduces the FL security and privacy anomalies. Next, we detail the existing security and privacy anomalies and the methods of detection and defense from the perspectives of the client, server, and communication process. Finally, we address the security and privacy anomalies in non-independent identically distributed cases during FL and summarize the related research progress. This survey aims to provide a systematic and comprehensive review of security and privacy research in FL to help understand the progress and better apply FL in additional scenarios.

show abstract

Efficient Detection of Byzantine Attacks in Federated Learning Using Last Layer Biases

Cited by 8 publications

References 6 publications

FL-Defender: Combating Targeted Attacks in Federated Learning

FL-Defender: Combating Targeted Attacks in Federated Learning

Federated Deep Learning for Cyber Security in the Internet of Things: Concepts, Applications, and Experimental Analysis

Anomaly detection and defense techniques in federated learning: a comprehensive review

Contact Info

Product

Resources

About