Context-Aware Online Client Selection for Hierarchical Federated Learning

Qu, Zhe; Duan, Rui; Chen, Lixing; Xu, Jie; Liu, Zhuo; Liu, Yao

doi:10.48550/arxiv.2112.00925

Cited by 2 publications

(2 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(2) deadline based FL architecture. [47] develops clients selection algorithm for deadline based HFL via contextual combinatorial multiarmed bandits to improve the training performance. (3) physical layer quantization.…”

Section: Related Workmentioning

confidence: 99%

On the Convergence of Multi-Server Federated Learning With Overlapping Area

et al. 2023

IEEE Trans. on Mobile Comput.

Self Cite

View full text Add to dashboard Cite

Multi-server Federated learning (FL) has been considered as a promising solution to address the limited communication resource problem of single-server FL. We consider a typical multi-server FL architecture, where the coverage areas of regional servers may overlap. The key point of this architecture is that the clients located in the overlapping areas update their local models based on the average model of all accessible regional models, which enables indirect model sharing among different regional servers. Due to the complicated network topology, the convergence analysis is much more challenging than single-server FL. In this paper, we firstly propose a novel MS-FedAvg algorithm for this multi-server FL architecture and analyze its convergence on non-iid datasets for general non-convex settings. Since the number of clients located in each regional server is much less than single-server FL, the bandwidth of each client should be large enough to successfully communicate training models with the server, which indicates that full client participation can work in multi-server FL. Also, we provide the convergence analysis of the partial client participation scheme and develop a new biased partial participation strategy to further accelerate convergence. Our results indicate that the convergence results highly depend on the ratio of the number of clients in each area type to the total number of clients in all three strategies. The extensive experiments show remarkable performance and support our theoretical results.

show abstract

Section: Related Workmentioning

confidence: 99%

On the Convergence of Multi-Server Federated Learning With Overlapping Area

et al. 2023

IEEE Trans. on Mobile Comput.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The challenges arise when training the joint model in FL from the non-i.i.d distributed training dataset, which firstly causes the problem of modeling the heterogeneity. In literature, there exists a large body of methods that models the statistical heterogeneity, (e.g., meta-learning [50], asynchronous learning [33] and multi-task learning [51]) which has been extended into the FL field, such as [13], [20], [52]- [54]. Additionally, the statistical heterogeneity of FL also causes problems on both the empirical performance and the convergence guarantee, even when learning a single joint model.…”

Section: Related Workmentioning

confidence: 99%

FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation

Li¹,

Qu²,

Tang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Federated Learning (FL) is a decentralized machine learning architecture, which leverages a large number of remote devices to learn a joint model with distributed training data. However, the system-heterogeneity is one major challenge in a FL network to achieve robust distributed learning performance, which is of two aspects: i) device-heterogeneity due to the diverse computational capacity among devices; ii) data-heterogeneity due to the non-identically distributed data across the network. Though there have been benchmarks against the heterogeneous FL, e.g., FedProx, the prior studies lack formalization and it remains an open problem. In this work, we formalize the system-heterogeneous FL problem and propose a new algorithm, called FedLGA, which addresses this problem by bridging the divergence of local model updates via gradient approximation. To achieve this, FedLGA provides an alternated Hessian estimation method, which only requires extra linear complexity on the aggregator. Theoretically, we show that with a device-heterogeneous ratio ρ, FedLGA achieves convergence rates on non-i.i.d distributed FL training data against non-convex optimization problems for O (1+ρ)

show abstract

Context-Aware Online Client Selection for Hierarchical Federated Learning

Cited by 2 publications

References 28 publications

On the Convergence of Multi-Server Federated Learning With Overlapping Area

On the Convergence of Multi-Server Federated Learning With Overlapping Area

FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation

Contact Info

Product

Resources

About