FRuDA: Framework for Distributed Adversarial Domain Adaptation

Gan, Shaoduo; Mathur, Akhil; Isopoussu, Anton; Kawsar, Fahim; Bianchi-Berthouze, Nadia; Lane, Nicholas D.

doi:10.1109/tpds.2021.3136673

Cited by 2 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Addressing distribution shifts is a key problem in FL, while most existing works focusing on label distribution skew through techniques such as training robust global models (Li et al, 2018c(Li et al, , 2021 or variance reduction methods (Karimireddy et al, 2020b,a). As another line of research, studies about feature distribution skew in FL mostly focus on domain generalization to train models that can generalize to unseen feature distributions (Peng et al, 2019;Wang et al, 2022a;Shen et al, 2021;Sun et al, 2022;Gan et al, 2021). All of the above methods aim to train a single robust model.…”

Section: Related Workmentioning

confidence: 99%

“…Many studies focus on adapting DG algorithms for FL scenarios. For example, combining FL with Distribution Robust Optimization (DRO), resulting in robust models that perform well on all clients (Mohri et al, 2019;Deng et al, 2021); combining FL with techniques that learn domain invariant features (Peng et al, 2019;Wang et al, 2022a;Shen et al, 2021;Sun et al, 2022;Gan et al, 2021) to improve the generalization ability of trained models. All of the above methods aim to train a single robust feature extractor that can generalize well on unseen distributions.…”

Section: A Proof Of Em Stepsmentioning

confidence: 99%

See 1 more Smart Citation

FedConceptEM: Robust Federated Learning Under Diverse Distribution Shifts

Guo¹,

Tang²,

Lin³

2023

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) is a machine learning paradigm that protects privacy by keeping client data on edge devices. However, optimizing FL in practice can be challenging due to the diversity and heterogeneity of the learning system. Recent research efforts have aimed to improve the optimization of FL with distribution shifts, but it is still an open problem how to train FL models when multiple types of distribution shifts, i.e., feature distribution skew, label distribution skew, and concept shift occur simultaneously. To address this challenge, we propose a novel algorithm framework, FedConceptEM, for handling diverse distribution shifts in FL. FedConceptEM automatically assigns clients with concept shifts to different models, avoiding the performance drop caused by these shifts. At the same time, clients without concept shifts, even with feature or label skew, are assigned to the same model, improving the robustness of the trained models. Extensive experiments demonstrate that FedConceptEM outperforms other state-of-the-art cluster-based FL methods by a significant margin.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: A Proof Of Em Stepsmentioning

confidence: 99%

FedConceptEM: Robust Federated Learning Under Diverse Distribution Shifts

Guo¹,

Tang²,

Lin³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Reisizadeh et al (2020) assumes the local distribution is perturbed by an affine function, i.e., from x to Ax + b. There are also some methods that aim to learn client invariant features (Peng et al, 2019;Wang et al, 2022;Sun et al, 2022;Gan et al, 2021). However, these methods are designed to learn a model that can perform well on unseen deployment distributions that different from (seen) clients' local distributions, which is out of our scope.…”

Section: Related Workmentioning

confidence: 99%

FedAug: Reducing the Local Learning Bias Improves Federated Learning on Heterogeneous Data

Guo¹,

Tang²,

Lin³

2022

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) is a machine learning paradigm that learns from data kept locally to safeguard the privacy of clients, whereas local SGD is typically employed on the clients' devices to improve communication efficiency. However, such a scheme is currently constrained by the slow and unstable convergence induced by clients' heterogeneous data. In this work, we identify three under-explored phenomena of the biased local learning that may explain these challenges caused by local updates in supervised FL. As a remedy, we propose FedAug, a novel unified algorithm that reduces the local learning bias on features and classifiers to tackle these challenges. FedAug consists of two components: AugMean and AugCA. AugMean alleviates the bias in the local classifiers by balancing the output distribution of models. AugCA learns client invariant features that are close to global features but considerably distinct from those learned from other input distributions. In a series of experiments, we show that FedAug consistently outperforms other SOTA FL and domain generalization (DG) baselines, in which both two components (i.e., AugMean and AugCA) have individual performance gains.

show abstract

FRuDA: Framework for Distributed Adversarial Domain Adaptation

Cited by 2 publications

References 22 publications

FedConceptEM: Robust Federated Learning Under Diverse Distribution Shifts

FedConceptEM: Robust Federated Learning Under Diverse Distribution Shifts

FedAug: Reducing the Local Learning Bias Improves Federated Learning on Heterogeneous Data

Contact Info

Product

Resources

About