Federated Residual Learning

Agarwal, Alekh; Langford, John; Wei, Chen-Yu

doi:10.48550/arxiv.2003.12880

Cited by 7 publications

(11 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to space limitations, we only present a subset of recovered personalized FL approaches here while the personalized FL with explicit weight sharing (Arivazhagan et al, 2019;Liang et al, 2020), federated residual learning (Agarwal et al, 2020), andMAML based approaches (Fallah et al, 2020) are discussed in the appendix.…”

Section: Personalized Fl Objectivesmentioning

confidence: 99%

See 1 more Smart Citation

Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

Hanzely¹,

Zhao²,

Kolar³

2021

Preprint

View full text Add to dashboard Cite

We study the optimization aspects of personalized Federated Learning (FL). We develop a universal optimization theory applicable to all convex personalized FL models in the literature. In particular, we propose a general personalized objective capable of recovering essentially any existing personalized FL objective as a special case. We design several optimization techniques to minimize the general objective, namely a tailored variant of Local SGD and variants of accelerated coordinate descent/accelerated SVRCD. We demonstrate the practicality and/or optimality of our methods both in terms of communication and local computation. Lastly, we argue about the implications of our general optimization theory when applied to solve specific personalized FL objectives.

show abstract

Section: Personalized Fl Objectivesmentioning

confidence: 99%

“…C.3. Federated residual learning (Agarwal et al, 2020) The last among the personalized FL models we mention is the federated residual learning from (Agarwal et al, 2020) given as:…”

Section: Objectivementioning

confidence: 99%

Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

Hanzely¹,

Zhao²,

Kolar³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…They both train an extra gating function for deciding which regions to trust one over another. [2] adopts a similar training process as ours. Each party privately trains the local model.…”

Section: Personalized Federated Learningmentioning

confidence: 99%

Practical and Secure Federated Recommendation with Personalized Masks

Liu¹,

Tan²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated recommendation is a new notion of private distributed recommender systems. It aims to address the data silo and privacy problems altogether. Current federated recommender systems mainly utilize homomorphic encryption and differential privacy methods to protect the intermediate computational results. However, the former comes with extra communication and computation costs, the latter damages model accuracy. Neither of them could simultaneously satisfy the real-time feedback and accurate personalization requirements of recommender systems. In this paper, we proposed a new federated recommendation framework, named federated masked matrix factorization. Federated masked matrix factorization could protect the data privacy in federated recommender systems without sacrificing efficiency or efficacy. Instead of using homomorphic encryption and differential privacy, we utilize the secret sharing technique to incorporate the secure aggregation process of federated matrix factorization. Compared with homomorphic encryption, secret sharing largely speeds up the whole training process. In addition, we introduce a new idea of personalized masks and apply it in the proposed federated masked matrix factorization framework. On the one hand, personalized masks could further improve efficiency. On the other hand, personalized masks also benefit efficacy. Empirically, we show the superiority of the designed model on different real-world data sets. Besides, we also provide the privacy guarantee and discuss the extension of the personalized mask method to the general federated learning tasks. CCS Concepts: • Information systems → Recommender systems; • Computing methodologies → Mobile agents; • Security and privacy → Distributed systems security.

show abstract

“…; Dubey and Pentland (2020) assume IID local models and focus on privacy protection. Agarwal et al (2020) studies regression-based contextual bandits as an example of the federated residual learning framework, which does not generalize to our formulation. The recent studies in Zhu et al (2020); Shi and Shen (2021) are more related to this work, where federated MAB without personalization (i.e., global-only) is studied.…”

Section: Related Workmentioning

confidence: 99%

“…In both applications, the general FL characteristics need to be applied to an underlying bandit model, which greatly complicates the problem. The bandit setting is more difficult due to limited feedback (only observing one arm at a time) (Agarwal et al, 2020). In addition, FL has a deterministic pipeline, while the data collection for bandit is online and the server-clients coordination becomes dynamic.…”

Section: Introductionmentioning

confidence: 99%

Federated Multi-armed Bandits with Personalization

Shi,

Shen,

Yang

2021

Preprint

View full text Add to dashboard Cite

A general framework of personalized federated multi-armed bandits (PF-MAB) is proposed, which is a new bandit paradigm analogous to the federated learning (FL) framework in supervised learning and enjoys the features of FL with personalization. Under the PF-MAB framework, a mixed bandit learning problem that flexibly balances generalization and personalization is studied. A lower bound analysis for the mixed model is presented. We then propose the Personalized Federated Upper Confidence Bound (PF-UCB) algorithm, where the exploration length is chosen carefully to achieve the desired balance of learning the local model and supplying global information for the mixed learning objective. Theoretical analysis proves that PF-UCB achieves an O(log(T )) regret regardless of the degree of personalization, and has a similar instance dependency as the lower bound. Experiments using both synthetic and real-world datasets corroborate the theoretical analysis and demonstrate the effectiveness of the proposed algorithm.

show abstract

Federated Residual Learning

Cited by 7 publications

References 0 publications

Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

Practical and Secure Federated Recommendation with Personalized Masks

Federated Multi-armed Bandits with Personalization

Contact Info

Product

Resources

About