Two-Stream Federated Learning: Reduce the Communication Costs

Yao, Xin; Chaofeng, Huang; Sun, Lifeng

doi:10.1109/vcip.2018.8698609

Cited by 95 publications

(47 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors of [117] also note that the biggest issues in FL are mainly security and privacy. As such, efficient FL algorithms that deliver models with high performance and privacy protection without adding computational burden are desirable [118], [119]. Because local models are trained via newer data to highlight new updates, it is likely that adversaries can influence the local training data-sets to compromise the models' results.…”

Section: Challenges and Limitations Of Federated Learningmentioning

confidence: 99%

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

et al. 2020

View full text Add to dashboard Cite

This paper provides a comprehensive study of Federated Learning (FL) with an emphasis on enabling software and hardware platforms, protocols, real-life applications and use-cases. FL can be applicable to multiple domains but applying it to different industries has its own set of obstacles. FL is known as collaborative learning, where algorithm(s) get trained across multiple devices or servers with decentralized data samples without having to exchange the actual data. This approach is radically different from other more established techniques such as getting the data samples uploaded to servers or having data in some form of distributed infrastructure. FL on the other hand generates more robust models without sharing data, leading to privacy-preserved solutions with higher security and access privileges to data. This paper starts by providing an overview of FL. Then, it gives an overview of technical details that pertain to FL enabling technologies, protocols, and applications. Compared to other survey papers in the field, our objective is to provide a more thorough summary of the most relevant protocols, platforms, and real-life use-cases of FL to enable data scientists to build better privacy-preserving solutions for industries in critical need of FL. We also provide an overview of key challenges presented in the recent literature and provide a summary of related research work. Moreover, we explore both the challenges and advantages of FL and present detailed service use-cases to illustrate how different architectures and protocols that use FL can fit together to deliver desired results.

show abstract

Section: Challenges and Limitations Of Federated Learningmentioning

confidence: 99%

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Third, Sahu et al [55] (and similarly Yao et al [75]) propose including an additional regularisation term during local training such that the solution space for weights is close to the global weights of the last epoch.…”

Section: Non-iid Datamentioning

confidence: 99%

“…FL requires clients to repeatedly send their model parameter updates and in return receive the new global parameters. Especially when there is a large number of clients (>100) involved in training, the communication efficiency becomes the main bottleneck of FL to achieve a quick model convergence [8,22,43,50,52,59,62,75,78,103]. This motivates a branch of FL research looking into ways to improve the efficiency of information exchange between the participants.…”

Section: Communication Efficiencymentioning

confidence: 99%

See 1 more Smart Citation

Federated Learning in a Medical Context: A Systematic Literature Review

Pfitzner

Steckhan

Arnrich

2021

ACM Trans. Internet Technol.

128

View full text Add to dashboard Cite

Data privacy is a very important issue. Especially in fields like medicine, it is paramount to abide by the existing privacy regulations to preserve patients’ anonymity. However, data is required for research and training machine learning models that could help gain insight into complex correlations or personalised treatments that may otherwise stay undiscovered. Those models generally scale with the amount of data available, but the current situation often prohibits building large databases across sites. So it would be beneficial to be able to combine similar or related data from different sites all over the world while still preserving data privacy. Federated learning has been proposed as a solution for this, because it relies on the sharing of machine learning models, instead of the raw data itself. That means private data never leaves the site or device it was collected on. Federated learning is an emerging research area, and many domains have been identified for the application of those methods. This systematic literature review provides an extensive look at the concept of and research into federated learning and its applicability for confidential healthcare datasets.

show abstract

“…Additionally, communication efficiency is a key issue of federated learning at present. Yao et al [ 28 ] presented a two-stream model rather than a single model for training to alleviate resource constraints with the maximum mean discrepancy principle each iteration. Vogels et al [ 29 ] introduced a novel low-rank gradient compression for power layer iteration to aggregate models rapidly and perform wall-clock speedups.…”

Section: Related Workmentioning

confidence: 99%

FedMed: A Federated Learning Framework for Language Modeling

Liang

Wang

2020

Sensors

View full text Add to dashboard Cite

Federated learning (FL) is a privacy-preserving technique for training a vast amount of decentralized data and making inferences on mobile devices. As a typical language modeling problem, mobile keyboard prediction aims at suggesting a probable next word or phrase and facilitating the human-machine interaction in a virtual keyboard of the smartphone or laptop. Mobile keyboard prediction with FL hopes to satisfy the growing demand that high-level data privacy be preserved in artificial intelligence applications even with the distributed models training. However, there are two major problems in the federated optimization for the prediction: (1) aggregating model parameters on the server-side and (2) reducing communication costs caused by model weights collection. To address the above issues, traditional FL methods simply use averaging aggregation or ignore communication costs. We propose a novel Federated Mediation (FedMed) framework with the adaptive aggregation, mediation incentive scheme, and topK strategy to address the model aggregation and communication costs. The performance is evaluated in terms of perplexity and communication rounds. Experiments are conducted on three datasets (i.e., Penn Treebank, WikiText-2, and Yelp) and the results demonstrate that our FedMed framework achieves robust performance and outperforms baseline approaches.

show abstract

Two-Stream Federated Learning: Reduce the Communication Costs

Cited by 95 publications

References 5 publications

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications

Federated Learning in a Medical Context: A Systematic Literature Review

FedMed: A Federated Learning Framework for Language Modeling

Contact Info

Product

Resources

About