ChannelFed: Enabling Personalized Federated Learning via Localized Channel Attention

Zheng, Kaiyu; Li, Xuefeng; Zhu, Guogang; Wu, Xinghao; Niu, Jianwei

doi:10.1109/globecom48099.2022.10000892

Cited by 4 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Concretely, the attention value is computed according to the average reward, average loss, training data size, etc, increasing the possibility of obtaining a more powerful agent model after aggregation. The attention-based module is also widely used for personalized federated learning [21,22]. In [22], the authors design a PFL framework termed ChannelFed that uses an attention module to assign different weights to channels on the client side.…”

Section: Attentive Aggregationmentioning

confidence: 99%

“…The attention-based module is also widely used for personalized federated learning [21,22]. In [22], the authors design a PFL framework termed ChannelFed that uses an attention module to assign different weights to channels on the client side. After incorporating personalized channel attention, the performance of the local model can be improved and client-specific knowledge can be better captured.…”

Section: Attentive Aggregationmentioning

confidence: 99%

See 1 more Smart Citation

Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning

Ji,

Tan,

Saravirta

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated learning is a new learning paradigm that decouples data collection and model training via multi-party computation and model aggregation.As a flexible learning setting, federated learning has the potential to integrate with other learning frameworks.We conduct a focused survey of federated learning in conjunction with other learning algorithms. Specifically, we explore various learning algorithms to improve the vanilla federated averaging algorithm and review model fusion methods such as adaptive aggregation, regularization, clustered methods, and Bayesian methods. Following the emerging trends, we also discuss federated learning in the intersection with other learning paradigms, termed federated X learning, where X includes multitask learning, meta-learning, transfer learning, unsupervised learning, and reinforcement learning. This survey reviews the state of the art, challenges, and future directions.

show abstract

Section: Attentive Aggregationmentioning

confidence: 99%

Section: Attentive Aggregationmentioning

confidence: 99%

Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning

Ji,

Tan,

Saravirta

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Consequences of FL failure include: (1) clients being unwilling to participate in FL, (2) wasted rounds of client computation (and client-server interactions), and (3) disintegration of the entire federation in the worst case. Many remedy solutions have been proposed to prevent FL failure [9,12,[17][18][19][20][21][22][23][24][25][26][27]. However, the FL system using the existing solutions faces a dilemma: If the remedy is predetermined to be used, it incurs extra (high) costs even if FL could have done well without such a remedy.…”

Section: Introductionmentioning

confidence: 99%

FL-GUARD: A Holistic Framework for Run-Time Detection and Recovery of Negative Federated Learning

Lin,

Shou,

Chen

et al. 2024

Data Sci. Eng.

View full text Add to dashboard Cite

Federated learning (FL) is a promising approach for learning a model from data distributed on massive clients without exposing data privacy. It works effectively in the ideal federation where clients share homogeneous data distribution and learning behavior. However, FL may fail to function appropriately when the federation is not ideal, amid an unhealthy state called Negative Federated Learning (NFL), in which most clients gain no benefit from participating in FL. Many studies have tried to address NFL. However, their solutions either (1) predetermine to prevent NFL in the entire learning life-cycle or (2) tackle NFL in the aftermath of numerous learning rounds. Thus, they either (1) indiscriminately incur extra costs even if FL can perform well without such costs or (2) waste numerous learning rounds. Additionally, none of the previous work takes into account the clients who may be unwilling/unable to follow the proposed NFL solutions when using those solutions to upgrade an FL system in use. This paper introduces FL-GUARD, a holistic framework that can be employed on any FL system for tackling NFL in a run-time paradigm. That is, to dynamically detect NFL at the early stage (tens of rounds) of learning and then to activate recovery measures when necessary. Specifically, we devise a cost-effective NFL detection mechanism, which relies on an estimation of performance gain on clients. Only when NFL is detected, we activate the NFL recovery process, in which each client learns in parallel an adapted model when training the global model. Extensive experiment results confirm the effectiveness of FL-GUARD in detecting NFL and recovering from NFL to a healthy learning state. We also show that FL-GUARD is compatible with previous NFL solutions and robust against clients unwilling/unable to take any recovery measures.

show abstract

Emerging trends in federated learning: from model fusion to federated X learning

Ji,

Tan,

Saravirta

et al. 2024

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

Federated learning is a new learning paradigm that decouples data collection and model training via multi-party computation and model aggregation. As a flexible learning setting, federated learning has the potential to integrate with other learning frameworks. We conduct a focused survey of federated learning in conjunction with other learning algorithms. Specifically, we explore various learning algorithms to improve the vanilla federated averaging algorithm and review model fusion methods such as adaptive aggregation, regularization, clustered methods, and Bayesian methods. Following the emerging trends, we also discuss federated learning in the intersection with other learning paradigms, termed federated X learning, where X includes multitask learning, meta-learning, transfer learning, unsupervised learning, and reinforcement learning. In addition to reviewing state-of-the-art studies, this paper also identifies key challenges and applications in this field, while also highlighting promising future directions.

show abstract

ChannelFed: Enabling Personalized Federated Learning via Localized Channel Attention

Cited by 4 publications

References 12 publications

Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning

Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning

FL-GUARD: A Holistic Framework for Run-Time Detection and Recovery of Negative Federated Learning

Emerging trends in federated learning: from model fusion to federated X learning

Contact Info

Product

Resources

About