Joint Optimization of Data Sampling and User Selection for Federated Learning in the Mobile Edge Computing Systems

Feng, Chenyuan; Wang, Yidong; Zhao, Zhongyuan; Quek, Tony Q. S.

doi:10.1109/iccworkshops49005.2020.9145182

Cited by 31 publications

(11 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The channel uncertainty is considered in [8] where the joint user scheduling and resource block allocation is performed so as to minimize the loss of FL accuracy. The cost and learning loss are jointly minimized in [18] by selecting mobile users who are participating in FL. The selected users are allowed to determine the amount of data samples used for the model training.…”

Section: A Related Work 1) Resource Management In Flmentioning

confidence: 99%

Energy-Aware Resource Management for Federated Learning in Multi-Access Edge Computing Systems

et al. 2021

View full text Add to dashboard Cite

In Federated Learning (FL), a global statistical model is developed by encouraging mobile users to perform the model training on their local data and aggregating the output local model parameters in an iterative manner. However, due to limited energy and computation capability at the mobile devices, the performance of the model training is always at stake to meet the objective of local energy minimization. In this regard, Multi-access Edge Computing (MEC)-enabled FL addresses the tradeoff between the model performance and the energy consumption of the mobile devices by allowing users to offload a portion of their local dataset to an edge server for the model training. Since the edge server has high computation capability, the time consumption of the model training at the edge server is insignificant. However, the time consumption for dataset offloading from mobile users to the edge server has a significant impact on the total time consumed to complete a single round of FL process. Thus, resource management in MEC-enabled FL is challenging, where the objective is to reduce the total time consumption while saving the energy consumption of the mobile devices. In this article, we formulate an energy-aware resource management for MEC-enabled FL in which the model training loss and the total time consumption are jointly minimized, while considering the energy limitation of mobile devices. In addition, we recast the formulated problem as a Generalized Nash Equilibrium Problem (GNEP) to capture the coupling constraints between the radio resource management and dataset offloading. To that end, we analyze the impact of the dataset offloading and computing resource allocation on the model training loss, time, and the energy consumption. Finally, we present the convergence analysis of the proposed solution, and evaluate its performance against the traditional FL approach. Simulation results demonstrate the efficacy of our proposed solution approach.INDEX TERMS Dataset offloading, energy-aware resource management, federated learning, generalized Nash equilibrium game, multi-access edge computing.

show abstract

Section: A Related Work 1) Resource Management In Flmentioning

confidence: 99%

Energy-Aware Resource Management for Federated Learning in Multi-Access Edge Computing Systems

et al. 2021

View full text Add to dashboard Cite

show abstract

“…This work has been extended in [217], where a control scheme is proposed, based on reinforcement learning, to accelerate the FL process by actively selecting the best subset of users in each communication round that can counterbalance the bias introduced by non-IID data. In [218], a joint optimization framework for sample selection and user selection was studied to keep a balance between the model accuracy and cost. However, the distribution distance between different users was optimized in this framework by adjusting the local batch size, which might lead to the under-utilization of data in strongly skewed users.…”

Section: Va Profiling Computation and Communication Modelsmentioning

confidence: 99%

Pervasive AI for IoT Applications: Resource-efficient Distributed Artificial Intelligence

Baccour,

Mhaisen,

Abdellatif

et al. 2021

Preprint

View full text Add to dashboard Cite

Artificial intelligence (AI) has witnessed a substantial breakthrough in a variety of Internet of Things (IoT) applications and services, spanning from recommendation systems and speech processing applications to robotics control and military surveillance. This is driven by the easier access to sensory data and the enormous scale of pervasive/ubiquitous devices that generate zettabytes (ZB) of real-time data streams. Designing accurate models using such data streams, to predict future insights and revolutionize the decision-taking process, inaugurates pervasive systems as a worthy paradigm for a better quality-oflife. The confluence of pervasive computing and artificial intelligence, Pervasive AI, expanded the role of ubiquitous IoT systems from mainly data collection to executing distributed computations with a promising alternative to centralized learning, presenting various challenges, including privacy concerns, scalability, and latency requirements. In this context, a wise cooperation and resource scheduling should be envisaged among IoT devices (e.g., smartphones, smart vehicles) and infrastructure (e.g. edge nodes, and base stations) to avoid communication and computation overheads and ensure maximum performance. In this paper, we conduct a comprehensive survey of the recent techniques and strategies developed to overcome these resource challenges in pervasive AI systems. Specifically, we first present an overview of the pervasive computing, its architecture, and its intersection with artificial intelligence. We then review the background, applications and performance metrics of AI, particularly Deep Learning (DL) and online learning, running in a ubiquitous system. Next, we provide a deep literature review of communication-efficient techniques, from both algorithmic and system perspectives, of distributed inference, training and online learning tasks across the combination of IoT devices, edge devices and cloud servers. Finally, we discuss our future vision and research challenges.

show abstract

“…Then, given the limited wireless resources, a joint learning, resource allocation, and user selection optimization problem was formulated to minimize the FL convergence time and training loss. In [15], a joint optimization framework for sample selection and user selection was studied to keep a balance between the model accuracy and cost. However, the distribution distance between different users was optimized in this framework through adjusting the local batch size, which might lead to the under-utilization of data in strongly skewed users.…”

Section: Single-layer Federated Learningmentioning

confidence: 99%

Communication-Efficient Hierarchical Federated Learning for IoT Heterogeneous Systems with Imbalanced Data

Abdellatif¹,

Mhaisen²,

Mohamed³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a distributed learning methodology that allows multiple nodes to cooperatively train a deep learning model, without the need to share their local data. It is a promising solution for telemonitoring systems that demand intensive data collection from different locations while maintaining a strict privacy constraint. Due to privacy concerns and critical communication bottlenecks, it can become impractical to send the FL updated models to a centralized server. Thus, this paper studies the potential of hierarchical FL in IoT heterogeneous systems and propose an optimized solution for user assignment and resource allocation on multiple edge nodes. In particular, this work focuses on a generic class of machine learning models that are trained using gradient-descent-based schemes while considering the practical constraints of non-uniformly distributed data across different users. We evaluate the proposed system using two real-world datasets, and we show that it outperforms state-of-the-art FL solutions. In particular, our numerical results highlight the effectiveness of our approach and its ability to provide 4-6% increase in the classification accuracy, with respect to hierarchical FL schemes that consider distance-based user assignment. Furthermore, the proposed approach could significantly accelerate FL training and reduce communication overhead by providing 75-85% reduction in the communication rounds between edge nodes and the centralized server, for the same model accuracy.

show abstract

Joint Optimization of Data Sampling and User Selection for Federated Learning in the Mobile Edge Computing Systems

Cited by 31 publications

References 5 publications

Energy-Aware Resource Management for Federated Learning in Multi-Access Edge Computing Systems

Energy-Aware Resource Management for Federated Learning in Multi-Access Edge Computing Systems

Pervasive AI for IoT Applications: Resource-efficient Distributed Artificial Intelligence

Communication-Efficient Hierarchical Federated Learning for IoT Heterogeneous Systems with Imbalanced Data

Contact Info

Product

Resources

About