A Triple-Step Asynchronous Federated Learning Mechanism for Client Activation, Interaction Optimization, and Aggregation Enhancement

You, Linlin; Liu, Sheng; Chang, Yi; Yuen, Chau

doi:10.1109/jiot.2022.3188556

Cited by 36 publications

(21 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In simple terms, the client parameters are sent to the central server after the local data may have undergone multiple epochs of local training. The parameter server then immediately aggregates the model parameters without waiting for any other clients and then returns the aggregated parameters to the client [10]. That is, when the server receives any update parameters from the client, an aggregation is done.…”

Section: Methods Based On Asynchronous Aggregationmentioning

confidence: 99%

“…There are many algorithms for federated learning based on server-side aggregation optimization, i.e., optimizing model aggregation, so there are also many directions for federated aggregation optimization. From the perspective of asynchronous aggregation, such as AsyncFedAvg [10], FedAsync [11]; from the perspective of hierarchical aggregation, such as FedPER [12], FedMA [13]; there are also server-side optimization methods, such as FedAvgM [14], which introduces momentum in the server, and adaptive server optimization methods such as FedAdagrad [15], FedYogi [15], and FedAdam [15]. There are other optimization methods that will not be discussed in this paper.…”

Section: Improvement On Aggregationmentioning

confidence: 99%

See 1 more Smart Citation

Federated learning on non-independent and identically distributed data

Hao-wei

Luo

Wang³

2023

Third International Conference on Machine Learning and Computer Application (ICMLCA 2022)

View full text Add to dashboard Cite

Section: Methods Based On Asynchronous Aggregationmentioning

confidence: 99%

Section: Improvement On Aggregationmentioning

confidence: 99%

Federated learning on non-independent and identically distributed data

Hao-wei

Luo

Wang³

2023

Third International Conference on Machine Learning and Computer Application (ICMLCA 2022)

View full text Add to dashboard Cite

“…In contrast to synchronous FL, the asynchronous scheme leads to faster convergence under unstable networks especially with millions of devices [28]. An increasing number of asynchronous FL works have been published in recent years, with focuses on client selection [17,24,29,63,64], weight aggregation [52,53,60] and transmission scheduling [35]. Semi-asynchronous mechanisms are developed to aggregate buffered updates [14,16,45,54,61].…”

Section: Related Workmentioning

confidence: 99%

Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks

Yu¹,

Cherkasova²,

Vardhan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) has gained increasing interest in recent years as a distributed on-device learning paradigm. However, multiple challenges remain to be addressed for deploying FL in real-world Internet-of-Things (IoT) networks with hierarchies. Although existing works have proposed various approaches to account data heterogeneity, system heterogeneity, unexpected stragglers and scalibility, none of them provides a systematic solution to address all of the challenges in a hierarchical and unreliable IoT network. In this paper, we propose an asynchronous and hierarchical framework (Async-HFL) for performing FL in a common three-tier IoT network architecture. In response to the largely varied delays, Async-HFL employs asynchronous aggregations at both the gateway and the cloud levels thus avoids long waiting time. To fully unleash the potential of Async-HFL in converging speed under system heterogeneities and stragglers, we design device selection at the gateway level and device-gateway association at the cloud level. Device selection chooses edge devices to trigger local training in real-time while device-gateway association determines the network topology periodically after several cloud epochs, both satisfying bandwidth limitation. We evaluate Async-HFL's convergence speedup using large-scale simulations based on ns-3 and a network topology from NYCMesh. Our results show that Async-HFL converges 1.08-1.31x faster in wall-clock time and saves up to 21.6% total communication cost compared to state-of-the-art asynchronous FL algorithms (with client selection). We further validate Async-HFL on a physical deployment and observe robust convergence under unexpected stragglers.

show abstract

“…Furthermore, the willingness-to-pay [44] utility equations are used to define the common estimation task as presented in Equations ( 23)- (25).…”

Section: Data and Model Structurementioning

confidence: 99%

“…As a novel solution to address the above challenges, federated learning (FL) [22][23][24] is now in the spotlight to learn AI models in a collaborative and privacy-preserving manner. Instead of being transmitted directly to the server, raw data are processed locally at each client for the learning parameters (i.e., gradients), which will be uploaded and aggregated at the server to generate a global model [25]. If such a process can be implemented in PMS, data islands and idle resources can be bridged and utilized to assist the learning of the intelligent core.…”

Section: Introductionmentioning

confidence: 99%

A Federated Mixed Logit Model for Personal Mobility Service in Autonomous Transportation Systems

You

Zhao

et al. 2022

Systems

Self Cite

View full text Add to dashboard Cite

Looking ahead to the future-stage autonomous transportation system (ATS), personal mobility service (PMS) aims to provide the recommended travel options based on both microscopic individual travel demand and the macroscopic supply system objectives. Such a goal relies on massive heterogeneous data to interpret and predict user travel intentions, facing the challenges caused by prevalent centralized approaches, such as an unbalanced utilization rate between cloud and edge, and data privacy. To fill the gap, we propose a federated logit model (FMXL), for estimating user preferences, which integrates a discrete choice model—the mixed logit model (MXL), with a novel decentralized learning paradigm—federated learning (FL). FMXL supports PMS by (1) respectively performing local and global estimation at the client and server to optimize the load, (2) collaboratively approximating the posterior of the standard mixed logit model through a continuous interaction mechanism, and (3) flexibly configuring two specific global estimation methods (sampling and aggregation) to accommodate different estimation scenarios. Moreover, the predicted rates of FMXL are about 10% higher compared to a flat logit model in both static and dynamic estimation. Meanwhile, the estimation time has been reduced by about 40% compared to a centralized MXL model. Our model can not only protect user privacy and improve the utilization of edge resources but also significantly improve the accuracy and timeliness of recommendations, thus enhancing the performance of PMS in ATS.

show abstract

A Triple-Step Asynchronous Federated Learning Mechanism for Client Activation, Interaction Optimization, and Aggregation Enhancement

Cited by 36 publications

References 26 publications

Federated learning on non-independent and identically distributed data

Federated learning on non-independent and identically distributed data

Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks

A Federated Mixed Logit Model for Personal Mobility Service in Autonomous Transportation Systems

Contact Info

Product

Resources

About