Energy-Efficient Radio Resource Allocation for Federated Edge Learning

Zeng, Qunsong; Du, Youwei; Leung, Kin K.; Huang, Kaibin

doi:10.48550/arxiv.1907.06040

Cited by 30 publications

(53 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The inherent trade-off between local model update and global model aggregation is studied in [32] to optimize over transmission power/rate and training time. Various radio resource allocation and client selection policies [12], [16]- [18], [33]- [35] have been proposed to minimize the learning loss or the training time. Joint communication and computation is investigated [13], [15], [30], [36], [37].…”

Section: Related Workmentioning

confidence: 99%

“…More recent research starts to fill this gap by focusing on the communication system design, particularly for wireless FL [10], [11] where the underlying communication is unreliable; see Section II for an overview. Nevertheless, the focus has been on resource allocation [12]- [14], device selection [15]- [18], or either uplink or downlink (but not both) cellular system design [13], [19].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Federated Learning Over Noisy Channels: Convergence Analysis and Design Examples

Wei

Shen

2022

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

Does Federated Learning (FL) work when both uplink and downlink communications have errors?How much communication noise can FL handle and what is its impact to the learning performance?This work is devoted to answering these practically important questions by explicitly incorporating both uplink and downlink noisy channels in the FL pipeline. We present several novel convergence analyses of FL over simultaneous uplink and downlink noisy communication channels, which encompass full and partial clients participation, direct model and model differential transmissions, and non-independent and identically distributed (IID) local datasets. These analyses characterize the sufficient conditions for FL over noisy channels to have the same convergence behavior as the ideal case of no communication error. More specifically, in order to maintain the O(1/T ) convergence rate of FEDAVG with perfect communications, the uplink and downlink signal-to-noise ratio (SNR) for direct model transmissions should be controlled such that they scale as O(t 2 ) where t is the index of communication rounds, but can stay constant for model differential transmissions. The key insight of these theoretical results is a "flying under the radar" principle -stochastic gradient descent (SGD) is an inherent noisy process and uplink/downlink communication noises can be tolerated as long as they do not dominate the time-varying SGD noise. We exemplify these theoretical findings with two widely adopted communication techniques -transmit power control and diversity combining -and further validating their performance advantages over the standard methods via extensive numerical experiments using several real-world FL tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Federated Learning Over Noisy Channels: Convergence Analysis and Design Examples

Wei

Shen

2022

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

show abstract

“…For example, a popular line of work is to reduce the communication load per worker by compression of the updates under the assumptions of ideal communication links, such as exploiting coding schemes [19], utilizing the sparsity of updates [20], employing quantization of the updates [21], and avoiding less informative local updates via communication censoring schemes [22]- [26]. Another line of work is to support FL through communication resource management, such as worker scheduling schemes to maximize the number of participating workers [27], joint optimization of resource allocation and worker scheduling [7], and communication and computation resource allocation and scheduling for cellfree networks [8]. There are some pioneering works on analog aggregation based FL [13]- [18], most of which focus on designing transmission schemes [13]- [16].…”

Section: Related Workmentioning

confidence: 99%

Joint Optimization of Communications and Federated Learning Over the Air

Fan,

Wang,

Huo

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is an attractive paradigm for making use of rich distributed data while protecting data privacy. Nonetheless, nonideal communication links and limited transmission resources have become the bottleneck of the implementation of fast and accurate FL. In this paper, we study joint optimization of communications and FL based on analog aggregation transmission in realistic wireless networks. We first derive a closed-form expression for the expected convergence rate of FL over the air, which theoretically quantifies the impact of analog aggregation on FL. Based on the analytical result, we develop a joint optimization model for accurate FL implementation, which allows a parameter server to select a subset of workers and determine an appropriate power scaling factor. Since the practical setting of FL over the air encounters unobservable parameters, we reformulate the joint optimization of worker selection and power allocation using controlled approximation. Finally, we efficiently solve the resulting mixed-integer programming problem via a simple yet optimal finite-set search method by reducing the search space. Simulation results show that the proposed solutions developed for realistic wireless analog channels outperform a benchmark method, and achieve comparable performance of the ideal case where FL is implemented over noise-free wireless channels.

show abstract

“…In [5], [13], the authors developed the federated edge learning algorithm DRAFT that schedules entries of the gradient vector based on the channel condition. Energy-efficiency aspects have been studied in [6]. Quantization methods of gradient transmissions were developed in [10].…”

Section: A Related Workmentioning

confidence: 99%

Accelerated Gradient Descent Learning over Multiple Access Fading Channels

Paul¹,

Friedman²,

Cohen³

2021

Preprint

View full text Add to dashboard Cite

We consider a distributed learning problem in a wireless network, consisting of N distributed edge devices and a parameter server (PS). The objective function is a sum of the edge devices' local loss functions, who aim to train a shared model by communicating with the PS over multiple access channels (MAC). This problem has attracted a growing interest in distributed sensing systems, and more recently in federated learning, known as over-the-air computation. In this paper, we develop a novel Accelerated Gradient-descent Multiple Access (AGMA) algorithm that uses momentum-based gradient signals over noisy fading MAC to improve the convergence rate as compared to existing methods. Furthermore, AGMA does not require power control or beamforming to cancel the fading effect, which simplifies the implementation complexity. We analyze AGMA theoretically, and establish a finite-sample bound of the error for both convex and strongly convex loss functions with Lipschitz gradient. For the strongly convex case, we show that AGMA approaches the best-known linear convergence rate as the network increases. For the convex case, we show that AGMA significantly improves the sub-linear convergence rate as compared to existing methods. Finally, we present simulation results using real datasets that demonstrate better performance by AGMA.

show abstract

Energy-Efficient Radio Resource Allocation for Federated Edge Learning

Cited by 30 publications

References 1 publication

Federated Learning Over Noisy Channels: Convergence Analysis and Design Examples

Federated Learning Over Noisy Channels: Convergence Analysis and Design Examples

Joint Optimization of Communications and Federated Learning Over the Air

Accelerated Gradient Descent Learning over Multiple Access Fading Channels

Contact Info

Product

Resources

About