Federated learning with adaptive communication compression under dynamic bandwidth and unreliable networks

Zhang, Xiongtao; Zhu, Xiaomin; Ji, Wang; Yan, Hui; Chen, Huangke; Bao, Weidong

doi:10.1016/j.ins.2020.05.137

Cited by 36 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it might be unreasonable to assume that each communication link has the same loss rate since it varies from the duration and packet size of different packets in reality. Consequently, Zhang et al [110] design a new algorithm called ACFL to adaptively compress the information from the shared model based on the physical conditions of the current network, taking into account the drop rate and the total number of transmitted packets.…”

Section: Unreliable Networkmentioning

confidence: 99%

Towards Efficient Communications in Federated Learning: A Contemporary Survey

Zhen¹,

Mao²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

In the traditional distributed machine learning scenario, the user's private data is transmitted between nodes and a central server, which results in great potential privacy risks. In order to balance the issues of data privacy and joint training of models, federated learning (FL) is proposed as a special distributed machine learning with a privacy protection mechanism, which can realize multiparty collaborative computing without revealing the original data. However, in practice, FL faces many challenging communication problems. This review aims to clarify the relationship between these communication problems, and focus on systematically analyzing the research progress of FL communication work from three perspectives: communication efficiency, communication environment, and communication resource allocation. Firstly, we sort out the current challenges existing in the communications of FL. Secondly, we have compiled articles related to FL communications, and then describe the development trend of the entire field guided by the logical relationship between them. Finally, we point

show abstract

Section: Unreliable Networkmentioning

confidence: 99%

Towards Efficient Communications in Federated Learning: A Contemporary Survey

Zhen¹,

Mao²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…When the local model completes a model parameter update, the change in the model weight is recorded, and these recorded data are uploaded to the central server. In this process, the local model training process of each computing node is relatively independent, and communication is achieved through the coordination of the central server [29]. At the beginning of the next iteration, the local model uses the parameters download from the central server as the initial model parameters and then uses the constrained stochastic gradient descent method to update the local model.…”

Section: Differential Privacy Preprocessing Of the Local Datamentioning

confidence: 99%

A Clonal Selection Optimization System for Multiparty Secure Computing

et al. 2021

View full text Add to dashboard Cite

The innovation of the deep learning modeling scheme plays an important role in promoting the research of complex problems handled with artificial intelligence in smart cities and the development of the next generation of information technology. With the widespread use of smart interactive devices and systems, the exponential growth of data volume and the complex modeling requirements increase the difficulty of deep learning modeling, and the classical centralized deep learning modeling scheme has encountered bottlenecks in the improvement of model performance and the diversification of smart application scenarios. The parallel processing system in deep learning links the virtual information space with the physical world, although the distributed deep learning research has become a crucial concern with its unique advantages in training efficiency, and improving the availability of trained models and preventing privacy disclosure are still the main challenges faced by related research. To address these above issues in distributed deep learning, this research developed a clonal selective optimization system based on the federated learning framework for the model training process involving large-scale data. This system adopts the heuristic clonal selective strategy in local model optimization and optimizes the effect of federated training. First of all, this process enhances the adaptability and robustness of the federated learning scheme and improves the modeling performance and training efficiency. Furthermore, this research attempts to improve the privacy security defense capability of the federated learning scheme for big data through differential privacy preprocessing. The simulation results show that the proposed clonal selection optimization system based on federated learning has significant optimization ability on model basic performance, stability, and privacy.

show abstract

“…Federated learning [31], [32] is a new upcoming area of research in decentralized training in an edge-cloud setting i.e., there is a common shared model that is trained on millions of IoT devices using the local data that is present on the device itself. The updated model is then sent to the cloud for global model aggregation.…”

Section: Related Work and Motivationmentioning

confidence: 99%

A Novel Approach of IoT Stream Sampling and Model Update on the IoT Edge Device for Class Incremental Learning in an Edge-Cloud System

2021

View full text Add to dashboard Cite

With the exponential rise of the number of IoT devices, the amount of data being produced is massive. Thus, it is unfeasible to send all the raw data directly to the cloud for processing, especially for data that is high dimensional. Training deep learning models incrementally evolves the model over time and eliminates the need to statically training the models with all the data. However, the integration of class incremental learning and the Internet of Things (IoT) is a new concept and is not yet mature. In the context of IoT and deep learning, the transmission cost of data in the edge-cloud architecture is a challenge. We demonstrate a novel sample selection method that discards certain training images on the IoT edge device that reduces transmission cost and still maintains class incremental learning performance. It can be unfeasible to transmit all parameters of a trained model back to the IoT edge device. Therefore, we propose an algorithm to find only the useful parameters of a trained model in an efficient way to reduce the transmission cost from the cloud to the edge devices. Results show that our proposed methods can effectively perform class-incremental learning in an edge-cloud setting.

show abstract

Federated learning with adaptive communication compression under dynamic bandwidth and unreliable networks

Cited by 36 publications

References 9 publications

Towards Efficient Communications in Federated Learning: A Contemporary Survey

Towards Efficient Communications in Federated Learning: A Contemporary Survey

A Clonal Selection Optimization System for Multiparty Secure Computing

A Novel Approach of IoT Stream Sampling and Model Update on the IoT Edge Device for Class Incremental Learning in an Edge-Cloud System

Contact Info

Product

Resources

About