Towards Efficient Communications in Federated Learning: A Contemporary Survey

Abstract

In the traditional distributed machine learning scenario, the user's private data is transmitted between nodes and a central server, which results in great potential privacy risks. In order to balance the issues of data privacy and joint training of models, federated learning (FL) is proposed as a special distributed machine learning with a privacy protection mechanism, which can realize multiparty collaborative computing without revealing the original data. However, in practice, FL faces many challenging communication problems. This review aims to clarify the relationship between these communication problems, and focus on systematically analyzing the research progress of FL communication work from three perspectives: communication efficiency, communication environment, and communication resource allocation. Firstly, we sort out the current challenges existing in the communications of FL. Secondly, we have compiled articles related to FL communications, and then describe the development trend of the entire field guided by the logical relationship between them. Finally, we point

A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regret

“…In terms of communication cost, there is a large body of work developing communication efficient algorithms by either reducing communication frequency [9,10,11,12,13] or reducing message size [14,15,16,17,18] by employing techniques like quantization and sparsification. See [19] and [20] for a detailed survey of such approaches. However, as mentioned earlier, most of these works treat the two components of communication cost separately and hence focus on reducing only one of these two components.…”