Towards Mitigating Device Heterogeneity in Federated Learning via Adaptive Model Quantization

Abdelmoniem, Ahmed M.; Canini, Marco

doi:10.1145/3437984.3458839

Cited by 33 publications

(13 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We expect that the heterogeneity impact might be reduced by means of proportional load balancing during the selection phase [37], computation offloading techniques at the edge [50], adaptive compression of model for computation [3] and communication [2,4] or asynchronous mode of model updates [30].…”

Section: Discussionmentioning

confidence: 99%

Empirical analysis of federated learning in heterogeneous environments

Abdelmoniem

Papageorgiou

et al. 2022

Proceedings of the 2nd European Workshop on Machine Learning and Systems

Self Cite

View full text Add to dashboard Cite

Federated learning (FL) is becoming a popular paradigm for collaborative learning over distributed, private datasets owned by non-trusting entities. FL has seen successful deployment in production environments, and it has been adopted in services such as virtual keyboards, auto-completion, item recommendation, and several IoT applications. However, FL comes with the challenge of performing training over largely heterogeneous datasets, devices, and networks that are out of the control of the centralized FL server. Motivated by this inherent setting, we make a first step towards characterizing the impact of device and behavioral heterogeneity on the trained model. We conduct an extensive empirical study spanning close to 1.5K unique configurations on five popular FL benchmarks. Our analysis shows that these sources of heterogeneity have a major impact on both model performance and fairness, thus shedding light on the importance of considering heterogeneity in FL system design.

show abstract

Section: Discussionmentioning

confidence: 99%

Empirical analysis of federated learning in heterogeneous environments

Abdelmoniem

Papageorgiou

et al. 2022

Proceedings of the 2nd European Workshop on Machine Learning and Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this context, device heterogeneity results in performance degradation due to stragglers (i.e., slow workers) who slow down the training process [48], [49]. Several works tried to address this problem via the system and algorithmic solutions [20], [39], [46], [48]- [50]. In FL settings, the heterogeneity is sourced from other system artifacts and is not limited to the heterogeneity in device capabilities.…”

Section: Related Workmentioning

confidence: 99%

A Comprehensive Empirical Study of Heterogeneity in Federated Learning

Abdelmoniem

Papageorgiou

et al. 2023

IEEE Internet Things J.

View full text Add to dashboard Cite

Federated learning (FL) is becoming a popular paradigm for collaborative learning over distributed, private datasets owned by non-trusting entities. FL has seen successful deployment in production environments, and it has been adopted in services such as virtual keyboards, auto-completion, item recommendation, and several IoT applications. However, FL comes with the challenge of performing training over largely heterogeneous datasets, devices, and networks that are out of the control of the centralized FL server. Motivated by this inherent challenge, we aim to empirically characterize the impact of device and behavioral heterogeneity on the trained model. We conduct an extensive empirical study spanning nearly 1.5K unique configurations on five popular FL benchmarks. Our analysis shows that these sources of heterogeneity have a major impact on both model quality and fairness, causing up to 4.6× and 2.2× degradation in the quality and fairness, respectively, thus shedding light on the importance of considering heterogeneity in FL system design.

show abstract

“…Additionally, the effectiveness of SFL with privacy and resilience safeguards is assessed in more extensive experimental situations. • Heterogeneity: FL faces a considerable challenge when operating in various devices and data of the whole system [71][72][73]. Indeed, increasingly intelligent devices can connect to train the FL system.…”

Section: Challenges Of Federated Learningmentioning

confidence: 99%

“…Extensive experiments on MNIST, FashionMNIST, MedMNIST, and CIFAR-10 demonstrate that their suggested approaches can achieve satisfactory performance with guaranteed convergence and efficiently use all the resources available for training across different devices with lower communication cost than its homogeneous counterpart. Abdelmoniem, A.M., and Canini, M. [73] also concentrate on reducing the degree of device heterogeneity by suggesting AQFL, a straightforward and useful method that uses adaptive model quantization to homogenize the customers' computational resources. They assess AQFL using five standard FL metrics.…”

Section: Challenges Of Federated Learningmentioning

confidence: 99%

Insights into Multi-Model Federated Learning: An Advanced Approach for Air Quality Index Forecasting

Khoa

Dao

et al. 2022

Algorithms

View full text Add to dashboard Cite

The air quality index (AQI) forecast in big cities is an exciting study area in smart cities and healthcare on the Internet of Things. In recent years, a large number of empirical, academic, and review papers using machine learning (ML) for air quality analysis have been published. However, most of those studies focused on traditional centralized processing on a single machine, and there had been few surveys of federated learning (FL) in this field. This overview aims to fill this gap and provide newcomers with a broader perspective to inform future research on this topic, especially for the multi-model approach. In this survey, we went over the works that previous scholars have conducted in AQI forecast both in traditional ML approaches and FL mechanisms. Our objective is to comprehend previous research on AQI prediction including methods, models, data sources, achievements, challenges, and solutions applied in the past. We also convey a new path of using multi-model FL, which has piqued the computer science community’s interest recently.

show abstract

Towards Mitigating Device Heterogeneity in Federated Learning via Adaptive Model Quantization

Cited by 33 publications

References 13 publications

Empirical analysis of federated learning in heterogeneous environments

Empirical analysis of federated learning in heterogeneous environments

A Comprehensive Empirical Study of Heterogeneity in Federated Learning

Insights into Multi-Model Federated Learning: An Advanced Approach for Air Quality Index Forecasting

Contact Info

Product

Resources

About