Unequal error protection transmission for federated learning

Abstract

Communication has been recognized as one of the primary challenges of federated learning (FL), but the actual communication algorithm or protocol design is still rarely involved in the existing studies. In the paper, viewing the model exchange in FL as a special kind of traffic, an unequal error protection (UEP) scheme is designed based on multi‐rate channel coding and multi‐layer modulation for it. To answer the question of how to make error control for FL when the wireless channel is no longer simplified as a pipeline, this paper firstly theoretically analyzes the impact of transmission error on machine leanring (ML) model, which reveals that the dynamic range of the weights should be taken into consideration. Guided by the analysis, the UEP scheme is applied to FL in multiple perspectives including parameter, network and time. Furthermore, a UEP‐based adaptive coding method is developed for the case with dynamic signal‐to‐noise ratio (SNR) to ensure faster and more stable convergence of the FL model while saving as much bandwidth as possible. Comprehensive numerical simulation on several real‐world datasets verifies that the proposed UEP transmission schemes can indeed bring significant benefits in accuracy, robustness and efficiency, especially when the channel condition is poor.

Federated Learning Based on Data Divergence and Differential Privacy in燜inancial Risk Control Research

“…In reference [16], a sparse ternary compression (STC) method, which uses a compression framework to improve federal learning under resource constraints, was proposed to overcome this challenge. The problems of model convergence and data heterogeneity in joint learning can be addressed by improving the FedAvg algorithm [17][18][19]. At present, federal learning has been applied to many fields, including health care, car driving, and communication.…”