On Analog Gradient Descent Learning Over Multiple Access Fading Channels

Abstract

We consider a distributed learning problem over multiple access channel (MAC) using a large wireless network. The computation is made by the network edge and is based on received data from a large number of distributed nodes which transmit over a noisy fading MAC. The objective function is a sum of the nodes' local loss functions. This problem has attracted a growing interest in distributed sensing systems, and more recently in federated learning. We develop a novel Gradient-Based Multiple Access (GBMA) algorithm to solve the distributed learning problem over MAC. Specifically, the nodes transmit an analog function of the local gradient using common shaping waveforms and the network edge receives a superposition of the analog transmitted signals used for updating the estimate. GBMA does not require power control or beamforming to cancel the fading effect as in other algorithms, and operates directly with noisy distorted gradients. We analyze the performance of GBMA theoretically, and prove that it can approach the convergence rate of the centralized gradient descent (GD) algorithm in large networks. Specifically, we establish a finite-sample bound of the error for both convex and strongly convex loss functions with Lipschitz gradient. Furthermore, we provide energy scaling laws for approaching the centralized convergence rate as the number of nodes increases. Finally, experimental results support the theoretical findings, and demonstrate strong performance of GBMA using synthetic and real data.Tomer Sery is with the

Privacy for Free: Wireless Federated Learning via Uncoded Transmission With Adaptive Power Control

“…To start, we denote as y k = [y k . According to the definition of DP loss given in (12), for the k-th device, the privacy loss after T iterations can be represented as…”

Harnessing Wireless Channels for Scalable and Privacy-Preserving Federated Learning

“…1(b). With channel inversion, transmissions are only allowed when |h n,i | 2 ≥ ε, in order to avoid excessive transmit power due to the inversion [9]- [11]. The choice of ε is heuristic, hindering the convergence analysis of analog FL.…”

Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications

“…This channel inversion however consumes the transmit power inversely proportional to the channel gain, which is not viable for small h n under the limited edge device energy budget. For this reason, it is common to allow transmissions only when the channel gains exceed a certain threshold [110]- [112]. As discussed in Sec.…”