HFML: heterogeneous hierarchical federated mutual learning on non-IID data

Li, Yang; Li, Jie; Li, Kan

doi:10.1007/s10479-023-05203-x

Cited by 5 publications

(5 citation statements)

References 54 publications

(67 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A different line of work includes the work by [55] that attempts to achieve global cluster centers by generating synthetic data at server instead of using original data. Similarly [32] draws inspiration from differential privacy which is quite different from our work. Federated Client Clustering: In federated client clustering, clients are clustered together to intelligently choose a subset of clients for the client update step.…”

Section: Related Workmentioning

confidence: 92%

MFC: A Multishot Approach to Federated Data Clustering

Makkar,

Bhumika,

Jain

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

The work explores the federated data clustering problem. The primary goal is to perform k-means clustering of data distributed over multiple clients while preserving privacy during an exchange with the central server. Existing solutions to unsupervised federated data clustering are either computationally challenging or effective only in heterogeneous regimes, i.e., when the number of clusters per client (kz) is less than the total number of clusters (k) (specifically, kz ≤ k). Moreover, existing one-shot approaches assume that the information about kz is available for each client. In this paper, we propose two multi-shot approaches which we call MFC and MFCH, that perform well on both heterogeneous and non-heterogeneous regimes, i.e., are independent of the underlying client data distribution. Both MFC and MFCH stand out as they do not rely on prior knowledge about kz. We theoretically bound the closeness of the local centers obtained by MFC to that of the optimal global centers and prove that under some well-separability assumption, the centers will be close enough. MFCH improvises MFC by only sharing a single cluster center from each client, thus ensuring more privacy. Our theoretical analysis shows that when at least O(k2log k) clients are involved, centers obtained by MFCH will closely approximate optimal global centers. Experiments on synthetic and real-world datasets validate the proposed approaches’ efficacy showcasing lower objective costs in non-heterogeneous regimes while having comparable performance in heterogeneous regimes. In addition, as a byproduct MFC exhibits higher device-level fairness in terms of the individual objective cost compared to existing state-of-the-art algorithms. The code is publicly available at https://github.com/shivi98g/MFC.

show abstract

Section: Related Workmentioning

confidence: 92%

MFC: A Multishot Approach to Federated Data Clustering

Makkar,

Bhumika,

Jain

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…Li et al. [20] give a proof of Local SGD convergence in the case of non‐IID dataset and shows that Local SGD still performs well in terms of convergence rate even in the non‐IID case.…”

Section: Related Workmentioning

confidence: 99%

Federated learning privacy incentives: Reverse auctions and negotiations

Lyu

Zhang

Wang

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

The incentive mechanism of federated learning has been a hot topic, but little research has been done on the compensation of privacy loss. To this end, this study uses the Local SGD federal learning framework and gives a theoretical analysis under the use of differential privacy protection. Based on the analysis, a multi‐attribute reverse auction model is proposed to be used for user selection as well as payment calculation for participation in federal learning. The model uses a mixture of economic and non‐economic attributes in making choices for users and is transformed into an optimisation equation to solve the user choice problem. In addition, a post‐auction negotiation model that uses the Rubinstein bargaining model as well as optimisation equations to describe the negotiation process and theoretically demonstrate the improvement of social welfare is proposed. In the experimental part, the authors find that their algorithm improves both the model accuracy and the F1‐score values relative to the comparison algorithms to varying degrees.

show abstract

“…To avoid sharing additional information, CLIMB [69], Bal-anceFL [47], and FedLC [67] learn balanced federated model by reweighting each client's importance during aggregation based on the empirical loss, balanced class sampling with self-entropy regularization, and logits calibration with pairwise margins respectively. FedRS [70] limits classifier updates when there are missing classes. However, conforming the federated model to a balanced class distribution may be detrimental for some clients, e.g., being sub-optimal compared to their locally-learned models.…”

Section: B Class Imbalance Learningmentioning

confidence: 99%

“…From single-model FL, we include 1) local learning, where each client trains a model individually, 2) FedAvg [1] as a baseline comparison, and representative approaches including FedProx [32], MOON [23], CReRF [33], and the mostrecent state-of-the-art approaches such as BalanceFL [47], FedRS [70], and FedLC [67].…”

Section: Implementation Detailsmentioning

confidence: 99%

Customized Federated Learning for Multi-Source Decentralized Medical Image Classification

Wicaksana

Yan

Yang

et al. 2022

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

The performance of deep networks for medical image analysis is often constrained by limited medical data, which is privacy-sensitive. Federated learning (FL) alleviates the constraint by allowing different institutions to collaboratively train a federated model without sharing data. However, the federated model is often suboptimal with respect to the characteristics of each client's local data. Instead of training a single global model, we propose Customized FL (CusFL), for which each client iteratively trains a client-specific/private model based on a federated global model aggregated from all private models trained in the immediate previous iteration. Two overarching strategies employed by CusFL lead to its superior performance: 1) the federated model is mainly for feature alignment and thus only consists of feature extraction layers; 2) the federated feature extractor is used to guide the training of each private model. In that way, CusFL allows each client to selectively learn useful knowledge from the federated model to improve its personalized model. We evaluated CusFL on multi-source medical image datasets for the identification of clinically significant prostate cancer and the classification of skin lesions.

show abstract

HFML: heterogeneous hierarchical federated mutual learning on non-IID data

Cited by 5 publications

References 54 publications

MFC: A Multishot Approach to Federated Data Clustering

MFC: A Multishot Approach to Federated Data Clustering

Federated learning privacy incentives: Reverse auctions and negotiations

Customized Federated Learning for Multi-Source Decentralized Medical Image Classification

Contact Info

Product

Resources

About