Resource-Efficient Federated Learning With Non-IID Data: An Auction Theoretic Approach

Seo, Eunil; Niyato, Dusit; Elmroth, Erik

doi:10.1109/jiot.2022.3197317

Cited by 12 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning-based client selec-tion does not need this a priori information but can result in communication overhead [11]. Resource-based client selection does not necessarily aim for better handling of Non-IID data but can help in this regard [16], [17]. All methods have in common that maintaining one global model tends to produce a bias and overftting towards "good" clients.…”

Section: State-of-the-art Taxonomymentioning

confidence: 99%

Handling Non-IID Data in Federated Learning: An Experimental Evaluation Towards Unified Metrics

Haller,

Lenz,

Nachtigall

et al. 2023

2023 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf

View full text Add to dashboard Cite

Recent research has demonstrated that Non-Identically Distributed (Non-IID) data can negatively impact the performance of global models constructed in federated learning. To address this concern, multiple approaches have been developed. Nonetheless, previous research lacks a cohesive overview and fails to uniformly assess these strategies, resulting in challenges when comparing and choosing relevant options for real-world scenarios. This study presents a structured survey of cutting-edge techniques for handling the Non-IID data, accompanied by proposing a metric to develop a standardized approach for assessing data skew and its harmony with the appropriate approach. The fndings affrm the metric's suitability as a heuristic for assessing data skew in distributed datasets without having insight into client data, serving both scientifc and practical purposes and thus supporting the selection of handling strategies. This preliminary research establishes the foundation for discussing standardizing methodologies for evaluating data heterogeneity in federated learning.

show abstract

Section: State-of-the-art Taxonomymentioning

confidence: 99%

Handling Non-IID Data in Federated Learning: An Experimental Evaluation Towards Unified Metrics

Haller,

Lenz,

Nachtigall

et al. 2023

2023 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf

View full text Add to dashboard Cite

show abstract

“…Data sharing is advocated as a means to enhance performance resilience when confronted with non-IID datasets featuring highly skewed distributions [37]. Remarkable performance gains can be obtained using a minimal amount of shared data [1]. The proponents argue that when a minuscule amount of shared data (i.e., 0.8% of the total data) can drasti-FIGURE 3: The accuracy of four well-known model-driven learning methods -FedAvg, Fedprox, Scaffold, and Moon -was evaluated based on their performance with data distributions having α values of 0.005, 0.02, 0.04, and 10 (which represents IID data) using the CIFAR100 dataset.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Additionally, the use of a data-sharing technique significantly reduces model convergence time and improves the overall accuracy of the trained model. Notably, this method is highly efficient and requires minimal input costs to generate or purchase marginal data while yielding substantial outputs in terms of time and accuracy improvements, presented in [1].…”

Section: Introductionmentioning

confidence: 99%

MadFed: Enhancing Federated Learning With Marginal-Data Model Fusion

Seo,

Elmroth

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

As the demand for intelligent applications at the network edge grows, so does the need for effective federated learning (FL) techniques. However, FL often relies on non-identically and nonindependently distributed local datasets across end devices, which could result in considerable performance degradation. Prior solutions, such as model-driven approaches based on knowledge distillation, metalearning, and transfer learning, have provided some reprieve. However, their performance suffers under heterogeneous local datasets and highly skewed data distributions. To address these challenges, this study introduces the Marginal Data Fusion Federated Learning (MadFed) approach, a groundbreaking fusion of model-and data-driven methodologies. By utilizing marginal data, MadFed mitigates data distribution skewness, improves the maximum achievable accuracy, and reduces communication costs. Furthermore, the study demonstrates that the fusion of marginal data can significantly improve performance even with minimal data entries, such as a single entry. For instance, it provides up to a 15.4% accuracy increase and 70.4% communication cost savings when combined with established model-driven methodologies. Conversely, relying solely on these model-driven methodologies can result in poor performance, especially with highly skewed datasets. Significantly, MadFed extends its effectiveness across various FL algorithms and offers a unique method to augment label sets of end devices, thereby enhancing the utility and applicability of federated learning in real-world scenarios. The proposed approach is not only efficient but also adaptable and versatile, promising broader application and potential for widespread adoption in the field.

show abstract

“…Zhao et al [28] improved training on non-IID data by constructing a small, globally shared, uniformly distributed data subset for all clients. Similarly, Seo et al [29] mitigated the quality degradation problem in FL via data sharing, using an auction approach to effectively reduce the cost, while satisfying system requirements for maximizing model quality and resource efficiency. In [30], the authors assume that a small segment of clients are willing to share their datasets, and the server collects data from these clients in a centralized manner to aid in updating the global model.…”

Section: Data-based Approachesmentioning

confidence: 99%

An Optimization Method for Non-IID Federated Learning Based on Deep Reinforcement Learning

Meng,

Li,

et al. 2023

Sensors

View full text Add to dashboard Cite

Federated learning (FL) is a distributed machine learning paradigm that enables a large number of clients to collaboratively train models without sharing data. However, when the private dataset between clients is not independent and identically distributed (non-IID), the local training objective is inconsistent with the global training objective, which possibly causes the convergence speed of FL to slow down, or even not converge. In this paper, we design a novel FL framework based on deep reinforcement learning (DRL), named FedRLCS. In FedRLCS, we primarily improved the greedy strategy and action space of the double DQN (DDQN) algorithm, enabling the server to select the optimal subset of clients from a non-IID dataset to participate in training, thereby accelerating model convergence and reaching the target accuracy in fewer communication epochs. In simulation experiments, we partition multiple datasets with different strategies to simulate non-IID on local clients. We adopt four models (LeNet-5, MobileNetV2, ResNet-18, ResNet-34) on the four datasets (CIFAR-10, CIFAR-100, NICO, Tiny ImageNet), respectively, and conduct comparative experiments with five state-of-the-art non-IID FL methods. Experimental results show that FedRLCS reduces the number of communication rounds required by 10–70% with the same target accuracy without increasing the computation and storage costs for all clients.

show abstract

Resource-Efficient Federated Learning With Non-IID Data: An Auction Theoretic Approach

Cited by 12 publications

References 26 publications

Handling Non-IID Data in Federated Learning: An Experimental Evaluation Towards Unified Metrics

Handling Non-IID Data in Federated Learning: An Experimental Evaluation Towards Unified Metrics

MadFed: Enhancing Federated Learning With Marginal-Data Model Fusion

An Optimization Method for Non-IID Federated Learning Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About