Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning

Shi, Wenqi; Zhou, Sheng; Niu, Zhisheng; Jiang, Miao; Lu, Geng

doi:10.1109/twc.2020.3025446

Cited by 289 publications

(146 citation statements)

References 20 publications

Supporting

Mentioning

144

Contrasting

Order By: Relevance

“…Note that the S-CSI has to be used to the wireless channels with fast fading, such as the application scenarios of Internet of Vehicles (IoV) networks. From (11), we firstly write the channel conditions with the given latency threshold γ th as…”

Section: B S-csi Based Ba Schemementioning

confidence: 99%

“…In further, some other wireless techniques such as mobile edge computing can be incorporated into the FL networks to reduce the communication cost, in order to accelerate the convergence [7]- [10]. Recently, the effect of latency on the FL networks has been studied, where several bandwidth allocation schemes were proposed to enhance the system performance [11], [12]. However, the channel state information is seldom incorporated into the system design of FL networks in the existing works, which motivates the work in this paper.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

System Optimization of Federated Learning Networks With a Constrained Latency

Zhang

Xia

Fan

et al. 2022

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

This paper investigates a wireless federated learning (FL) network with limited communication bandwidth, where multiple mobile clients train their individual models with the help of one central server. We consider the practical communication scenarios, where the clients should complete the local computation and model upload within a defined latency. By jointly exploiting the dynamic characteristics of wireless channels and computational capability at the clients, we optimize the federated learning network by maximizing the number of active clients under the constraints of both latency and bandwidth. Specifically, we propose two bandwidth allocation (BA) schemes, where scheme I is based on the instantaneous channel state information (CSI), while scheme II employs the particle swarm optimization (PSO) method, based on the statistical CSI. Simulation results on the test accuracy and convergence rate are finally provided to demonstrate the advantages of the proposed optimization schemes for the considered FL network.

show abstract

Section: B S-csi Based Ba Schemementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

System Optimization of Federated Learning Networks With a Constrained Latency

Zhang

Xia

Fan

et al. 2022

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

show abstract

“…Under both protocols, they minimize the total energy consumption at all edge devices over a particular finite training duration subject to a given training accuracy, by jointly optimizing the transmission power and rates at the edge devices for uploading model parameters and their central processing unit frequencies for local update. In [32], the authors propose a joint device scheduling and resource allocation policy to maximize the model accuracy within a given total training time budget for latency constrained wireless FL. The work in [34] analyzes how to design dynamic FL in mobile edge networks that optimally chooses the number of selected clients and the number of local iterations in each training round to minimize the total cost while ensuring convergence.…”

Section: Related Workmentioning

confidence: 99%

Lyapunov-based Optimization of Edge Resources for Energy-Efficient Adaptive Federated Learning

Battiloro¹,

Lorenzo²,

Merluzzi³

et al. 2021

Preprint

View full text Add to dashboard Cite

The aim of this paper is to propose a novel dynamic resource allocation strategy for energy-efficient adaptive federated learning at the wireless network edge, with latency and learning performance guarantees. We consider a set of devices collecting local data and uploading processed information to an edge server, which runs stochastic gradient-based algorithms to perform continuous learning and adaptation. Hinging on Lyapunov stochastic optimization tools, we dynamically optimize radio parameters (e.g., set of transmitting devices, transmit powers, bits, and rates) and computation resources (e.g., CPU cycles at devices and at server) in order to strike the best trade-off between power, latency, and performance of the federated learning task. The framework admits both a model-based implementation, where the learning performance metrics are available in closed-form, and a data-driven approach, which works with online estimates of the learning performance of interest. The method is then customized to the case of federated least mean squares (LMS) estimation, and federated training of deep convolutional neural networks. Numerical results illustrate the effectiveness of our strategy to perform energy-efficient, low-latency, adaptive federated learning at the wireless network edge.

show abstract

“…Another line of works aims to minimize wall-clock time via resource-aware optimization-based approaches, such as CPU frequency allocation (e.g., [29]), and communication bandwidth allocation (e.g., [30], [31]), straggler-aware client scheduling (e.g., [20], [32]- [37]), parameters control (e.g., [36]- [39]), and task offloading (e.g., [40], [41]). While these papers provided some novel insights, their optimization approaches did not consider how client sampling affects the total wall-clock time and thus are orthogonal to our work.…”

Section: Related Workmentioning

confidence: 99%

Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling

Luo¹,

Xiao²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning (FL) algorithms usually sample a fraction of clients in each round (partial participation) when the number of participants is large and the server's communication bandwidth is limited. Recent works on the convergence analysis of FL have focused on unbiased client sampling, e.g., sampling uniformly at random, which suffers from slow wall-clock time for convergence due to high degrees of system heterogeneity and statistical heterogeneity. This paper aims to design an adaptive client sampling algorithm that tackles both system and statistical heterogeneity to minimize the wall-clock convergence time. We obtain a new tractable convergence bound for FL algorithms with arbitrary client sampling probabilities. Based on the bound, we analytically establish the relationship between the total learning time and sampling probabilities, which results in a non-convex optimization problem for training time minimization. We design an efficient algorithm for learning the unknown parameters in the convergence bound and develop a low-complexity algorithm to approximately solve the non-convex problem. Experimental results from both hardware prototype and simulation demonstrate that our proposed sampling scheme significantly reduces the convergence time compared to several baseline sampling schemes. Notably, our scheme in hardware prototype spends 73% less time than the uniform sampling baseline for reaching the same target loss.3 We use wall-clock time to distinguish from the number of training rounds.

show abstract

Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning

Cited by 289 publications

References 20 publications

System Optimization of Federated Learning Networks With a Constrained Latency

System Optimization of Federated Learning Networks With a Constrained Latency

Lyapunov-based Optimization of Edge Resources for Energy-Efficient Adaptive Federated Learning

Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling

Contact Info

Product

Resources

About