Survey of Personalization Techniques for Federated Learning

Kulkarni, Viraj; Kulkarni, Milind; Pant, Aniruddha

doi:10.48550/arxiv.2003.08673

Cited by 16 publications

(20 citation statements)

References 23 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is also a line of work using model-agnostic meta learning [Finn et al, 2017] to achieve personalization [Jiang et al, 2019, Fallah et al, 2020. Other strategies have been proposed (see, e.g., Arivazhagan et al 2019, Li and Wang 2019, Mansour et al 2020, Yu et al 2020), and we refer readers to Kulkarni et al [2020] for a comprehensive survey. We briefly remark here that all the papers mentioned above only consider the optimization properties of their proposed algorithms, while we focus on statistical properties of personalized federated learning.…”

Section: Related Workmentioning

confidence: 99%

“…This underlines the essence of personalized federated learning, which seeks to develop algorithms that perform well over a wide spectrum of data heterogeneity. Despite a venerable line of work on personalized federated learning (see, e.g., Kulkarni et al 2020), the literature remains relatively silent on how the fundamental limits of personalized federated learning depend on data heterogeneity, as opposed to two extreme cases where both the minimax optimal rates and algorithms are known.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Theorem of the Alternative for Personalized Federated Learning

Chen,

Zheng,

Long

et al. 2021

Preprint

View full text Add to dashboard Cite

A widely recognized difficulty in federated learning arises from the statistical heterogeneity among clients: local datasets often come from different but not entirely unrelated distributions, and personalization is, therefore, necessary to achieve optimal results from each individual's perspective. In this paper, we show how the excess risks of personalized federated learning with a smooth, strongly convex loss depend on data heterogeneity from a minimax point of view. Our analysis reveals a surprising theorem of the alternative for personalized federated learning: there exists a threshold such that (a) if a certain measure of data heterogeneity is below this threshold, the FedAvg algorithm [McMahan et al., 2017] is minimax optimal; (b) when the measure of heterogeneity is above this threshold, then doing pure local training (i.e., clients solve empirical risk minimization problems on their local datasets without any communication) is minimax optimal. As an implication, our results show that the presumably difficult (infinitedimensional) problem of adapting to client-wise heterogeneity can be reduced to a simple binary decision problem of choosing between the two baseline algorithms. Our analysis relies on a new notion of algorithmic stability that takes into account the nature of federated learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Theorem of the Alternative for Personalized Federated Learning

Chen,

Zheng,

Long

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In general, such improvements are complementary to FedMix and can be used to further enhance its performance. We refer the interested readers to the recent surveys by (Kairouz et al, 2019;Kulkarni et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Federated Mixture of Experts

Reisser,

Louizos,

Gavves

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning (FL) has emerged as the predominant approach for collaborative training of neural network models across multiple users, without the need to gather the data at a central location. One of the important challenges in this setting is data heterogeneity, i.e. different users have different data characteristics. For this reason, training and using a single global model might be suboptimal when considering the performance of each of the individual user's data. In this work, we tackle this problem via Federated Mixture of Experts, FedMix, a framework that allows us to train an ensemble of specialized models. FedMix adaptively selects and trains a user-specific selection of the ensemble members. We show that users with similar data characteristics select the same members and therefore share statistical strength while mitigating the effect of non-i.i.d data. Empirically, we show through an extensive experimental evaluation that FedMix improves performance compared to using a single global model across a variety of different sources of non-i.i.d.-ness.

show abstract

“…Although FL can preserve privacy for users in AI training, data features embedded in model updates can be leaked which thus can reveal private user information [75]. To overcome this challenge, a differential privacy technique is adopted by injecting artificial noise into the local gradient training in each communication round.…”

Section: Flchain For Edge Crowdsensingmentioning

confidence: 99%

Federated Learning Meets Blockchain in Edge Computing: Opportunities and Challenges

Nguyen¹,

Ding²,

Pham³

et al. 2021

Preprint

View full text Add to dashboard Cite

Mobile edge computing (MEC) has been envisioned as a promising paradigm to handle the massive volume of data generated from ubiquitous mobile devices for enabling intelligent services with the help of artificial intelligence (AI). Traditionally, AI techniques often require centralized data collection and training in a single entity, e.g., an MEC server, which is now becoming a weak point due to data privacy concerns and high data communication overheads. In this context, federated learning (FL) has been proposed to provide collaborative data training solutions, by coordinating multiple mobile devices to train a shared AI model without exposing their data, which enjoys considerable privacy enhancement. To improve the security and scalability of FL implementation, blockchain as a ledger technology is attractive for realizing decentralized FL training without the need for any central server. Particularly, the integration of FL and blockchain leads to a new paradigm, called FLchain, which potentially transforms intelligent MEC networks into decentralized, secure, and privacy-enhancing systems. This article presents an overview of the fundamental concepts and explores the opportunities of FLchain in MEC networks. We identify several main topics in FLchain design, including communication cost, resource allocation, incentive mechanism, security and privacy protection. The key solutions for FLchain design are provided, and the lessons learned as well as the outlooks are also discussed. Then, we investigate the applications of FLchain in popular MEC domains, such as edge data sharing, edge content caching and edge crowdsensing. Finally, important research challenges and future directions are also highlighted.

show abstract

Survey of Personalization Techniques for Federated Learning

Cited by 16 publications

References 23 publications

A Theorem of the Alternative for Personalized Federated Learning

A Theorem of the Alternative for Personalized Federated Learning

Federated Mixture of Experts

Federated Learning Meets Blockchain in Edge Computing: Opportunities and Challenges

Contact Info

Product

Resources

About