HiFlash: Communication-Efficient Hierarchical Federated Learning With Adaptive Staleness Control and Heterogeneity-Aware Client-Edge Association

Wu, Qiong; Chen, Xu; Ouyang, Tao; Zhou, Zhi; Zhang, Xiaoxi; Yang, Shusen; Zhang, Junshan

doi:10.1109/tpds.2023.3238049

Cited by 17 publications

(7 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The edge servers and the clients perform one-step gradient descent as (6) for τ + 1 intra-set iterations. Then, the edge servers forward the model parameters to the cloud server, and cloud aggregation is performed according to (10). The specialized optimality gap is given in the next corollary.…”

Section: B Convergence Analysismentioning

confidence: 99%

A Hessian-Based Federated Learning Approach to Tackle Statistical Heterogeneity

Ahmad,

Luo,

Robles-Kelly

2023

Advanced Data Mining and Applications

View full text Add to dashboard Cite

This paper presents a novel hierarchical federated learning algorithm within multiple sets that incorporates quantization for communication-efficiency and demonstrates resilience to statistical heterogeneity. Unlike conventional hierarchical federated learning algorithms, our approach combines gradient aggregation in intra-set iterations with model aggregation in inter-set iterations. We offer a comprehensive analytical framework to evaluate its optimality gap and convergence rate, comparing these aspects with those of conventional algorithms. Additionally, we develop a problem formulation to derive optimal system parameters in a closed-form solution. Our findings reveal that our algorithm consistently achieves high learning accuracy over a range of parameters and significantly outperforms other hierarchical algorithms, particularly in scenarios with heterogeneous data distributions.

show abstract

Section: B Convergence Analysismentioning

confidence: 99%

A Hessian-Based Federated Learning Approach to Tackle Statistical Heterogeneity

Ahmad,

Luo,

Robles-Kelly

2023

Advanced Data Mining and Applications

View full text Add to dashboard Cite

show abstract

“…Staleness in asynchronous SFL is further investigated in [21], which introduces an asynchronous update rule based on the notion that staleness effects in SFL are similar in consecutive training epochs. In [22] the connection bottleneck is tackled by combining synchronous orchestration with a dynamic aggregation rule that ignores stragglers.…”

Section: A Related Workmentioning

confidence: 99%

On-Board Federated Learning for Dense LEO Constellations

Razmi

Matthiesen

Dekorsy

et al. 2022

ICC 2022 - IEEE International Conference on Communications

View full text Add to dashboard Cite

The emergence of mega-constellations of interconnected satellites has a major impact on the integration of cellular wireless and non-terrestrial networks (NTN). This paper studies the problem of running a federated learning (FL) algorithm within a low Earth orbit (LEO) constellation of satellites connected with intra-orbit inter-satellite links (ISL). Satellites apply on-board machine learning and transmit the local parameters to the parameter server (PS). The main contribution is a novel approach to enhance FL in satellite constellations using intra-orbit ISLs. The key idea is to rely on the predictability of satellite visits to create a system design in which ISLs mitigate the impact of intermittent connectivity and transmit the aggregated parameters to the PS. We first devise a synchronous FL, which is then extended towards an asynchronous FL for the case of sparse satellite visits to the PS. An efficient use of the satellite resources is attained by sparsification-based compression the aggregated parameters of each orbit before forwarding to the PS. Performance is evaluated in terms of accuracy and the required size of data to be transmitted. The numerical results indicate a faster convergence rate of the presented approach compared with the state-of-the-art FL on satellite constellations.

show abstract

“…The edge server acts as an intermediate in the system by grouping the clients and fusing the machine learning models, which is referred to as edge-assisted FL. By preaggregating local models at the edge layer, the amount of data updated by the model can be reduced by an order of magnitude [6]. With the same bandwidth, the number of clients occupying the channel is reduced, and the available bandwidth is increased.…”

Section: Introductionmentioning

confidence: 99%

“…However, edge-assisted federated learning is constrained by the system heterogeneity, where the clients vary in their computing and communication capabilities; as well, data heterogeneity often happens in different clients with diverse local data. Furthermore, the bottleneck between communication from the cloud to clients is still a pressing issue [6]. For example, during the process of synchronous global model aggregation, the clients remain idle, so a significant amount of waiting time is wasted.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

Sha,

Sun,

Liu

et al. 2024

Electronics

View full text Add to dashboard Cite

Federated learning (FL) is widely regarded as highly promising because it enables the collaborative training of high-performance machine learning models among a large number of clients while preserving data privacy by keeping the data local. However, many existing FL frameworks have a two-layered architecture, thus requiring the frequent exchange of large-scale model parameters between clients and remote cloud servers over often unstable networks and resulting in significant communication overhead and latency. To address this issue, we propose to introduce edge servers between the clients and the cloud server to assist in aggregating local models, thus combining asynchronous client–edge model aggregation with synchronous edge–cloud model aggregation. By leveraging the clients’ idle time to accelerate training, the proposed framework can achieve faster convergence and reduce the amount of communication traffic. To make full use of the grouping properties inherent in three-layer FL, we propose a similarity matching strategy between edges and clients, thus improving the effect of asynchronous training. We further propose to introduce model-contrastive learning into the loss function and personalize the clients’ local models to address the potential learning issues resulting from asynchronous local training in order to further improve the convergence speed. Extensive experiments confirm that our method exhibits significant improvements in model accuracy and convergence speed when compared with other state-of-the-art federated learning architectures.

show abstract

HiFlash: Communication-Efficient Hierarchical Federated Learning With Adaptive Staleness Control and Heterogeneity-Aware Client-Edge Association

Cited by 17 publications

References 31 publications

A Hessian-Based Federated Learning Approach to Tackle Statistical Heterogeneity

A Hessian-Based Federated Learning Approach to Tackle Statistical Heterogeneity

On-Board Federated Learning for Dense LEO Constellations

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

Contact Info

Product

Resources

About