Federated Learning Over Noisy Channels: Convergence Analysis and Design Examples

Abstract

Does Federated Learning (FL) work when both uplink and downlink communications have errors?How much communication noise can FL handle and what is its impact to the learning performance?This work is devoted to answering these practically important questions by explicitly incorporating both uplink and downlink noisy channels in the FL pipeline. We present several novel convergence analyses of FL over simultaneous uplink and downlink noisy communication channels, which encompass full and partial clients participation, direct model and model differential transmissions, and non-independent and identically distributed (IID) local datasets. These analyses characterize the sufficient conditions for FL over noisy channels to have the same convergence behavior as the ideal case of no communication error. More specifically, in order to maintain the O(1/T ) convergence rate of FEDAVG with perfect communications, the uplink and downlink signal-to-noise ratio (SNR) for direct model transmissions should be controlled such that they scale as O(t 2 ) where t is the index of communication rounds, but can stay constant for model differential transmissions. The key insight of these theoretical results is a "flying under the radar" principle -stochastic gradient descent (SGD) is an inherent noisy process and uplink/downlink communication noises can be tolerated as long as they do not dominate the time-varying SGD noise. We exemplify these theoretical findings with two widely adopted communication techniques -transmit power control and diversity combining -and further validating their performance advantages over the standard methods via extensive numerical experiments using several real-world FL tasks.

Resource Rationing for Wireless Federated Learning: Concept, Benefits, and Challenges

“…6. Comparing the performance of transmit power control [12] to the baselines with partial clients participation, model transmission and IID dataset for FL on the CIFAR-10 dataset. A single-cell multi-user cellular system with broadband analog aggregation in [7] for FEDAVG is simulated where user devices participate in FL over wireless uplink and downlink communications.…”

“…The proposed resource rationing framework, however, operates on the time scale of communication rounds, which allows for the flexibility to incorporate existing or future "fast" resource allocation mechanisms or prediction methods to handle temporal variations. We illustrate this flexibility using transmit power control as a use case [12], and Fig. 6 illustrates the performance advantage of combining the "later-is-better" principle of resource rationing with an inner-loop power control that handles channel fading and interference.…”

“…Early research on communication-efficient FL largely focused on reducing the amount of information to be transmitted, and did not touch the actual communication algorithm and protocol design [3], [4]. More recent research started to fill this void from a wireless communication and networking point of view [5]- [12]. In general, the principle is to balance learning performance and communication efficiency via, e.g., device selection, bandwidth allocation, and power control.…”

Joint Device Selection and Power Control for Wireless Federated Learning

“…Therefore, broadcasting of inaccurate global model is also an important issue for the implementation of the FL algorithms. Without considering the wireless channels, the authors in [25] proposed a linear projection method to broadcast a compressed global model to the devices, and the authors in [26] derived the sufficient conditions for controlling the signal-to-noise ratios (SNRs) of both the downlink and uplink transmission to maintain the linear convergence rate of the FL algorithm [26].…”

“…Different from [25], [26] and under wireless implementations of FL, the authors in [27], [28] provided the convergence analysis of both the digital and analog downlink transmissions and showed the advantages of the analog downlink transmission, and the same result was presented in [29] numerically. To the best of our knowledge, there are few existing works addressing how to efficiently mitigate the impact of both the downlink and uplink wireless communications on the FL algorithm.…”