Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Data

Yoshida, Naoya; Nishio, Takayuki; Morikura, Masahiro; Yamamoto, Koji; Yonetani, Ryo

doi:10.1109/icc40277.2020.9149323

Cited by 124 publications

(68 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…otherwise (35) where 1) ). We prove (34) and (35) in Appendix D. Using these results, we have a closed form for the maximum expected total aggregate return from the nodes as follows:…”

Section: Analyzing Codedfedl Load Designmentioning

confidence: 99%

“…Our proposed coded computing framework, CodedFedL, provides a novel solution for leveraging coding redundancy for straggler resilient federated learning. Prior works that have considered one or more aspects of compute, communication and statistical heterogeneity across clients in federated learning include [32]- [35]. In [32], a Fed-Prox algorithm was proposed to address non-IID data across clients.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Coded Computing for Low-Latency Federated Learning Over Wireless Edge Networks

Prakash

Dhakal

Akdeniz

et al. 2021

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

Federated learning enables training a global model from data located at the client nodes, without data sharing and moving client data to a centralized server. Performance of federated learning in a multi-access edge computing (MEC) network suffers from slow convergence due to heterogeneity and stochastic fluctuations in compute power and communication link qualities across clients. We propose a novel coded computing framework, CodedFedL, that injects structured coding redundancy into federated learning for mitigating stragglers and speeding up the training procedure. CodedFedL enables coded computing for non-linear federated learning by efficiently exploiting distributed kernel embedding via random Fourier features that transforms the training task into computationally favourable distributed linear regression. Furthermore, clients generate local parity datasets by coding over their local datasets, while the server combines them to obtain the global parity dataset. Gradient from the global parity dataset compensates for straggling gradients during training, and thereby speeds up convergence. For minimizing the epoch deadline time at the MEC server, we provide a tractable approach for finding the amount of coding redundancy and the number of local data points that a client processes during training, by exploiting the statistical properties of compute as well as communication delays. We also characterize the leakage in data privacy when clients share their local parity datasets with the server. Additionally, we analyze the convergence rate and iteration complexity of CodedFedL under simplifying assumptions, by treating CodedFedL as a stochastic gradient descent algorithm. Finally, for demonstrating gains that CodedFedL can achieve in practice, we conduct numerical experiments using practical network parameters and benchmark datasets, in which CodedFedL speeds up the overall training time by up to 15× in comparison to the benchmark schemes.

show abstract

“…otherwise (35) where 1) ). We prove (34) and (35) in Appendix D. Using these results, we have a closed form for the maximum expected total aggregate return from the nodes as follows:…”

Section: Analyzing Codedfedl Load Designmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Coded Computing for Low-Latency Federated Learning Over Wireless Edge Networks

Prakash

Dhakal

Akdeniz

et al. 2021

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

show abstract

“…• Algorithm Optimization: optimize the FL algorithm and perform more computation on clients to reduce the convergence time by reducing the number of rounds on the expense of more computation [17]- [24]. • Selective Updates: select only important updates from the clients or select the best clients in regards to the clients' resources and data size [25]- [30]. • Model Compression: reduce the amount of data exchanged between clients and the server [18], [31]- [33].…”

Section: A Federated Learningmentioning

confidence: 99%

“…In [28] and [29], the authors proposed the selection of clients based on the consumed energy in model's transmission and training, clients' distance from the server, and channel availability using Deep Reinforcement Learning (DRL) approach. Yoshida et al [30] propose a hybrid FL approach based on the assumption that some clients share and upload their data to the server to improve the accuracy and mitigate the degradation resulted from non-independentand-identically-distributed (non-IID) data. However, uploading clients' data to the server violates the rules of FL.…”

Section: B Selective Updatesmentioning

confidence: 99%

Budgeted Online Selection of Candidate IoT Clients to Participate in Federated Learning

Mohammed

Tabatabai

Al-Fuqaha

et al. 2021

IEEE Internet Things J.

View full text Add to dashboard Cite

Machine Learning (ML), and Deep Learning (DL) in particular, play a vital role in providing smart services to the industry. These techniques however suffer from privacy and security concerns since data is collected from clients and then stored and processed at a central location. Federated Learning (FL), an architecture in which model parameters are exchanged instead of client data, has been proposed as a solution to these concerns. Nevertheless, FL trains a global model by communicating with clients over communication rounds, which introduces more traffic on the network and increases the convergence time to the target accuracy. In this work, we solve the problem of optimizing accuracy in stateful FL with a budgeted number of candidate clients by selecting the best candidate clients in terms of test accuracy to participate in the training process. Next, we propose an online stateful FL heuristic to find the best candidate clients. Additionally, we propose an IoT client alarm application that utilizes the proposed heuristic in training a stateful FL global model based on IoT device type classification to alert clients about unauthorized IoT devices in their environment. To test the efficiency of the proposed online heuristic, we conduct several experiments using a real dataset and compare the results against state-of-the-art algorithms. Our results indicate that the proposed heuristic outperforms the online random algorithm with up to 27% gain in accuracy. Additionally, the performance of the proposed online heuristic is comparable to the performance of the best offline algorithm.

show abstract

“…Another open problem of FL we would like to mention here is the client selection problem, originally proposed in [1] and followed by some related works (e.g. [2], [14]- [16]). Many of them see the problem from a communication perspective, focusing on building an efficient selection or bandwidth allocation scheme that helps shorten the communication length.…”

Section: Related Workmentioning

confidence: 99%

An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guarantee

Huang

Lin

et al. 2020

IEEE Trans. Parallel Distrib. Syst.

103

View full text Add to dashboard Cite

The issue of potential privacy leakage during centralized AI's model training has drawn intensive concern from the public. A Parallel and Distributed Computing (or PDC) scheme, termed Federated Learning (FL), has emerged as a new paradigm to cope with the privacy issue by allowing clients to perform model training locally, without the necessity to upload their personal sensitive data. In FL, the number of clients could be sufficiently large, but the bandwidth available for model distribution and re-upload is quite limited, making it sensible to only involve part of the volunteers to participate in the training process. The client selection policy is critical to an FL process in terms of training efficiency, the final model's quality as well as fairness. In this paper, we will model the fairness guaranteed client selection as a Lyapunov optimization problem and then a C 2 MAB-based method is proposed for estimation of the model exchange time between each client and the server, based on which we design a fairness guaranteed algorithm termed RBCS-F for problem-solving. The regret of RBCS-F is strictly bounded by a finite constant, justifying its theoretical feasibility. Barring the theoretical results, more empirical data can be derived from our real training experiments on public datasets.

show abstract

Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Data

Cited by 124 publications

References 7 publications

Coded Computing for Low-Latency Federated Learning Over Wireless Edge Networks

Coded Computing for Low-Latency Federated Learning Over Wireless Edge Networks

Budgeted Online Selection of Candidate IoT Clients to Participate in Federated Learning

An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guarantee

Contact Info

Product

Resources

About