Federated Learning via Over-the-Air Computation

Abstract

The stringent requirements for low-latency and privacy of the emerging high-stake applications with intelligent devices such as drones and smart vehicles make the cloud computing inapplicable in these scenarios. Instead, edge machine learning becomes increasingly attractive for performing training and inference directly at network edges without sending data to a centralized data center. This stimulates a nascent field termed as federated learning for training a machine learning model on computation, storage, energy and bandwidth limited mobile devices in a distributed manner. To preserve data privacy and address the issues of unbalanced and non-IID data points across different devices, the federated averaging algorithm has been proposed for global model aggregation by computing the weighted average of locally updated model at each selected device. However, the limited communication bandwidth becomes the main bottleneck for aggregating the locally computed updates. We thus propose a novel over-the-air computation based approach for fast global model aggregation via exploring the superposition property of a wireless multiple-access channel. This is achieved by joint device selection and beamforming design, which is modeled as a sparse and low-rank optimization problem to support efficient algorithms design. To achieve this goal, we provide a differenceof-convex-functions (DC) representation for the sparse and lowrank function to enhance sparsity and accurately detect the fixed-rank constraint in the procedure of device selection. A DC algorithm is further developed to solve the resulting DC program with global convergence guarantees. The algorithmic advantages and admirable performance of the proposed methodologies are demonstrated through extensive numerical results.

Scheduling Policies for Federated Learning in Wireless Networks

“…• At a typical UE k, it solves a local optimization problem (14) using only the data stored on the device. Based on the solution, the UE updates the local reference a t [k] per (18), and if being selected by the AP, it sends out a global update ∆v t k via the allocated subchannel. • At the AP side, it selects a subgroup of UEs for update collection, decodes the received packet, and performs a global aggregation according to (20).…”

Reconfigurable Intelligent Surface for Green Edge Inference in Machine Learning

“…However, such a relaxation may not be tight, i.e., the solution obtained by SDR may not satisfy the rank-one constraint. As pointed out in [45], [46], the performance of SDR degrades sharply as the problem size grows. In our case, when M and/or K is large, the probability of returning a rank-one solution is low.…”

Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence

“…The proposed BAA can dramatically reduce communication latency compared with traditional Orthogonal Frequency Division Multiple Access (OFDMA). [34] also explores the over-the-air computation for model aggregation in Federated Learning. Concretely, [34] puts the principle into practice by modeling the device selection and beamforming design as a sparse and low-rank optimization problem, which is highly intractably combinatorial.…”

“…[34] also explores the over-the-air computation for model aggregation in Federated Learning. Concretely, [34] puts the principle into practice by modeling the device selection and beamforming design as a sparse and low-rank optimization problem, which is highly intractably combinatorial. To solve the problem with fast convergence rate, this paper proposed a difference-of-convex-functions (DC) representation via successive convex relaxation.…”