Communication has been known to be one of the primary bottlenecks of federated learning (FL), and yet existing studies have not addressed the efficient communication design, particularly in wireless FL where both uplink and downlink communications have to be considered. In this paper, we focus on the design and analysis of physical layer quantization and transmission methods for wireless FL. We answer the question of what and how to communicate between clients and the parameter server and evaluate the impact of the various quantization and transmission options of the updated model on the learning performance. We provide new convergence analysis of the well-known FEDAVG under noni.i.d. dataset distributions, partial clients participation, and finite-precision quantization in uplink and downlink communications. These analyses reveal that, in order to achieve an O(1/T) convergence rate with quantization, transmitting the weight requires increasing the quantization level at a logarithmic rate, while transmitting the weight differential can keep a constant quantization level. Comprehensive numerical evaluation on various real-world datasets reveals that the benefit of a FL-tailored uplink and downlink communication design is enormous-a carefully designed quantization and transmission achieves more than 98% of the floating-point baseline accuracy with fewer than 10% of the baseline bandwidth, for majority of the experiments on both i.i.d. and non-i.i.d. datasets. In particular, 1-bit quantization (3.1% of the floating-point baseline bandwidth) achieves 99.8% of the floating-point baseline accuracy at almost the same convergence rate on MNIST, representing the best known bandwidth-accuracy tradeoff to the best of the authors' knowledge.

No abstract

Communication has been recognized as one of the primary challenges of federated learning (FL), but the actual communication algorithm or protocol design is still rarely involved in the existing studies. In the paper, viewing the model exchange in FL as a special kind of traffic, an unequal error protection (UEP) scheme is designed based on multi‐rate channel coding and multi‐layer modulation for it. To answer the question of how to make error control for FL when the wireless channel is no longer simplified as a pipeline, this paper firstly theoretically analyzes the impact of transmission error on machine leanring (ML) model, which reveals that the dynamic range of the weights should be taken into consideration. Guided by the analysis, the UEP scheme is applied to FL in multiple perspectives including parameter, network and time. Furthermore, a UEP‐based adaptive coding method is developed for the case with dynamic signal‐to‐noise ratio (SNR) to ensure faster and more stable convergence of the FL model while saving as much bandwidth as possible. Comprehensive numerical simulation on several real‐world datasets verifies that the proposed UEP transmission schemes can indeed bring significant benefits in accuracy, robustness and efficiency, especially when the channel condition is poor.

We advocate a new resource allocation framework, which we term resource rationing, for wireless federated learning (FL). Unlike existing resource allocation methods for FL, resource rationing focuses on balancing resources across learning rounds so that their collective impact on the federated learning performance is explicitly captured. This new framework can be integrated seamlessly with existing resource allocation schemes to optimize the convergence of FL. In particular, a novel "later-isbetter" principle is at the front and center of resource rationing, which is validated empirically in several instances of wireless FL. We also point out technical challenges and research opportunities that are worth pursuing. Resource rationing highlights the benefits of treating the emerging FL as a new class of service that has its own characteristics, and designing communication algorithms for this particular service.

No abstract

No abstract

No abstract