Broadband Analog Aggregation for Low-Latency Federated Edge Learning

Abstract

To leverage rich data distributed at the network edge, a new machine-learning paradigm, called edge learning, has emerged where learning algorithms are deployed at the edge for providing intelligent services to mobile users.While computing speeds are advancing rapidly, the communication latency is becoming the bottleneck of fast edge learning. To address this issue, this work is focused on designing a low-latency multi-access scheme for edge learning. To this end, we consider a popular privacy-preserving framework, federated edge learning (FEEL), where a global AI-model at an edge-server is updated by aggregating (averaging) local models trained at edge devices. It is proposed that the updates simultaneously transmitted by devices over broadband channels should be analog aggregated "over-the-air" by exploiting the waveform-superposition property of a multi-access channel.Such broadband analog aggregation (BAA) results in dramatical communication-latency reduction compared with the conventional orthogonal access (i.e., OFDMA). In this work, the effects of BAA on learning performance are quantified targeting a single-cell random network. First, we derive two tradeoffs between communication-andlearning metrics, which are useful for network planning and optimization. The power control ("truncated channel inversion") required for BAA results in a tradeoff between the update-reliability [as measured by the receive signalto-noise ratio (SNR)] and the expected update-truncation ratio. Consider the scheduling of cell-interior devices to constrain path loss. This gives rises to the other tradeoff between the receive SNR and fraction of data exploited in learning. Next, the latency-reduction ratio of the proposed BAA with respect to the traditional OFDMA scheme is proved to scale almost linearly with the device population. Experiments based on a neural network and a real dataset are conducted for corroborating the theoretical results. In addition, we discuss the extensions of BAA to acquire safety against adversary attacks and integrate beamforming for enhancing cell-edge links.

On Analog Gradient Descent Learning Over Multiple Access Fading Channels

“…The channel fading is mitigated at the receiver by using multiple antennas, where the fading diminishes as the number of antennas approaches infinity. In [13], [14], the authors considered transmissions over fading MAC, where each entry of the gradient vector is scheduled for transmission depending on the corresponding channel condition. They developed the federated edge learning (FEEL) algorithm, where each node updates the SGD estimate for multiple steps, and then communicates with the server for model aggregation.…”

Cooperative Learning VIA Federated Distillation OVER Fading Channels

“…is defined as (10). The PS scales the received signal (2) by the factor (11) and multiplies it by R T ρ /ρ to obtain an estimate of K k=1 v k i .…”

Age-Based Scheduling Policy for Federated Learning in Mobile Edge Networks

“…Recognizing such criticality, a host of studies have been carried out and resulted in various scheduling protocols for FL, ranging from minimizing the transmission latency [13], maximizing the spectral utility [14], to opportunistically alternating between selecting the UEs with advantageous and disadvantageous channel conditions [15]. Even though positive gains have been demonstrated, these works put the main focus on exploiting the spectral resources so as to maximize the number of updates collectible by the AP in each round of global communication but ignore the staleness of these updates.…”