Lyapunov-based Optimization of Edge Resources for Energy-Efficient Adaptive Federated Learning

Abstract

The aim of this paper is to propose a novel dynamic resource allocation strategy for energy-efficient adaptive federated learning at the wireless network edge, with latency and learning performance guarantees. We consider a set of devices collecting local data and uploading processed information to an edge server, which runs stochastic gradient-based algorithms to perform continuous learning and adaptation. Hinging on Lyapunov stochastic optimization tools, we dynamically optimize radio parameters (e.g., set of transmitting devices, transmit powers, bits, and rates) and computation resources (e.g., CPU cycles at devices and at server) in order to strike the best trade-off between power, latency, and performance of the federated learning task. The framework admits both a model-based implementation, where the learning performance metrics are available in closed-form, and a data-driven approach, which works with online estimates of the learning performance of interest. The method is then customized to the case of federated least mean squares (LMS) estimation, and federated training of deep convolutional neural networks. Numerical results illustrate the effectiveness of our strategy to perform energy-efficient, low-latency, adaptive federated learning at the wireless network edge.

Dynamic Resource Optimization for Adaptive Federated Learning Empowered by Reconfigurable Intelligent Surfaces

“…The aim of this paper is to jointly allocate radio (i.e., set of transmitting devices, powers, quantization bits and RISs reflectivity parameters) and computation (i.e., CPU cycles at devices and at server) resources to minimize the long-term average system power consumption in (7), with constraints on the average learning performance and the average latency in (6). Let Gt and αt be task-dependent learning performance and convergence rate metrics, respectively, which will be explained in the sequel (see, e.g., (9)). Then, the problem can be cast as:…”

“…The rationale for the selection of the surrogates comes from the assumption that the true performance metrics Gt and αt typically show a non-increasing behavior with respect to the quantization bits {bi,t}i∈S t and the batch size Bt (examples will be given in the sequel). In other words, a finer representation of the data typically leads to better learning performance [8,9].Thus, Gt and αt have only to be non-increasing functions of the quantization bits and the batch size. After some algebra manipulations (omitted due to the lack of space), the method requires to solve the following deterministic problem at each time-slot t:…”

“…The deployment of such a flexible and complex architecture will enable Machine Learning (ML) and Artificial Intelligence (AI) at the edge, a paradigm known as edge machine learning. The challenge is that edge ML calls for jointly optimizing inference, training, communication, computation, and control under end-to-end latency, reliability, and learning performance requirements [6][7][8][9]. In this context, the exploitation of RISs in this joint design plays a key role, since it can strongly mitigate the presence of poor wireless channel conditions due to mobility, time-varying environment, and blocking events, especially using millimeter wave communications [2,10].…”

“…Related works. Training ML models at the edge mainly relies on Federated Learning (FL) [9,[11][12][13][14][15]. In this approach, learning…”