FLOP: Federated Learning on Medical Datasets using Partial Networks

Yang, Qian; Zhang, Jianyi; Hao, Wen; Spell, Gregory P.; Carin, Lawrence

doi:10.1145/3447548.3467185

Cited by 50 publications

(35 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It successfully addresses the data island challenge without completely compromising clients' privacy [12,22]. Recently, it has attracted significant interests in academia and achieved remarkable successes in various industrial applications, e.g., autonomous driving [39], wearable devices [33], medical diagnosis [10,52] and mobile phones [36].…”

Section: Introductionmentioning

confidence: 99%

“…Generally, most existing FL methods [16,42,46,52] are modeled in a static application scenario, where data classes of the overall FL framework are fixed and known in advance. However, real-world applications are often dynamic, where local clients receive the data of new classes in an online manner.…”

Section: Introductionmentioning

confidence: 99%

“…However, real-world applications are often dynamic, where local clients receive the data of new classes in an online manner. To handle such a setting, existing FL methods typically require storing all training data of old classes at the local clients' side so that a global model can be obtained via FL, however the high storage and computation overhead may render the FL unrealistic when new classes arrive dynamically [33,36,47,52]. And if these methods [42,52] are required to learn new classes continuously with very limited storage memory, they may suffer from significant performance degradation (i.e., catastrophic forgetting [20,37,40]) on old classes.…”

Section: Introductionmentioning

confidence: 99%

“…To handle such a setting, existing FL methods typically require storing all training data of old classes at the local clients' side so that a global model can be obtained via FL, however the high storage and computation overhead may render the FL unrealistic when new classes arrive dynamically [33,36,47,52]. And if these methods [42,52] are required to learn new classes continuously with very limited storage memory, they may suffer from significant performance degradation (i.e., catastrophic forgetting [20,37,40]) on old classes. Moreover, in real-world scenarios, new local clients that collect the data of new classes in a streaming manner may want to participate in the FL training, which could further exacerbate the catastrophic forgetting on old classes in the global model training.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Federated Class-Incremental Learning

Dong¹,

Wang²,

Fang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) has attracted growing attentions via data-private collaborative training on decentralized clients. However, most existing methods unrealistically assume object classes of the overall framework are fixed over time. It makes the global model suffer from significant catastrophic forgetting on old classes in real-world scenarios, where local clients often collect new classes continuously and have very limited storage memory to store old classes. Moreover, new clients with unseen new classes may participate in the FL training, further aggravating the catastrophic forgetting of global model. To address these challenges, we develop a novel Global-Local Forgetting Compensation (GLFC) model, to learn a global classincremental model for alleviating the catastrophic forgetting from both local and global perspectives. Specifically, to address local forgetting caused by class imbalance at the local clients, we design a class-aware gradient compensation loss and a class-semantic relation distillation loss to balance the forgetting of old classes and distill consistent inter-class relations across tasks. To tackle the global forgetting brought by the non-i.i.d class imbalance across clients, we propose a proxy server that selects the best old global model to assist the local relation distillation. Moreover, a prototype gradient-based communication mechanism is developed to protect the privacy. Our model outperforms state-of-the-art methods by 4.4%∼15.1% in terms of average accuracy on representative benchmark datasets. The code is available at https://github. com/conditionWang/FCIL.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Federated Class-Incremental Learning

Dong¹,

Wang²,

Fang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Derived but different from the standard distributed optimization [13], federated learning (FL) [24,39,57] is tailored for data privacy protection and efficient distributed training. FL has been applied to many specific tasks, e.g., language modeling [10], medical relation extraction [49], industrial topic modeling [10], and medical image analysis [58].…”

Section: Introductionmentioning

confidence: 99%

Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenes

Li,

Gan,

Zhan

et al. 2021

Preprint

View full text Add to dashboard Cite

Although federated learning (FL) has recently been proposed for efficient distributed training and data privacy protection, it still encounters many obstacles. One of these is the naturally existing statistical heterogeneity among clients, making local data distributions non independently and identically distributed (i.e., non-iid), which poses challenges for model aggregation and personalization. For FL with a deep neural network (DNN), privatizing some layers is a simple yet effective solution for non-iid problems. However, which layers should we privatize to facilitate the learning process? Do different categories of non-iid scenes have preferred privatization ways? Can we automatically learn the most appropriate privatization way during FL? In this paper, we answer these questions via abundant experimental studies on several FL benchmarks. First, we present the detailed statistics of these benchmarks and categorize them into covariate and label shift non-iid scenes. Then, we investigate both coarse-grained and fine-grained network splits and explore whether the preferred privatization ways have any potential relations to the specific category of a non-iid scene. Our findings are exciting, e.g., privatizing the base layers could boost the performances even in label shift non-iid scenes, which are inconsistent with some natural conjectures. We also find that none of these privatization ways could improve the performances on the Shakespeare benchmark, and we guess that Shakespeare may not be a seriously non-iid scene. Finally, we propose several approaches to automatically learn where to aggregate via cross-stitch, soft attention, and hard selection. We advocate the proposed methods could serve as a preliminary try to explore where to privatize for a novel non-iid scene.

show abstract