Abstract:Federated learning (FL) is a promising and powerful approach for training deep learning models without sharing the raw data of clients. During the training process of FL, the central server and distributed clients need to exchange a vast amount of model information periodically. To address the challenge of communication-intensive training, we propose a new training method, referred to as federated learning with dualside low-rank compression (FedDLR), where the deep learning model is compressed via low-rank app… Show more
“…The server then averages all received sub-sampled updates to get an estimate of the global model parameters. Additionally, [52] used dual-side low-rank compression to reduce the size of the models in both directions between the server and the nodes. Finally, [36] used a layer-based parameter selection in order to transfer only the important parameters of each model's layer.…”
Federated Machine Learning (Fed ML) is a new distributed machine learning technique applied to collaboratively train a global model using clients' local data without transmitting it. Nodes only send parameter updates (e.g., weight updates in the case of neural networks), which are fused together by the server to build the global model. By not divulging node data, Fed ML guarantees its confidentiality, a crucial aspect of network security, which enables it to be used in the context of data-sensitive Internet of Things (IoT) and mobile applications, such as smart geo-location and the smart grid. However, most IoT devices are particularly energy constrained, which raises the need to optimize the Fed ML process for efficient training tasks and optimized power consumption. In this paper, we conduct, to the best of our knowledge, the first Systematic Mapping Study (SMS) on FedML optimization techniques for energy-constrained IoT devices. From a total of more than 800 papers, we select 67 that satisfy our criteria and give a structured overview of the field using a set of carefully chosen research questions. Finally, we attempt to provide an analysis of the energy-constrained Fed ML state of the art and try to outline some potential recommendations for the research community.
“…The server then averages all received sub-sampled updates to get an estimate of the global model parameters. Additionally, [52] used dual-side low-rank compression to reduce the size of the models in both directions between the server and the nodes. Finally, [36] used a layer-based parameter selection in order to transfer only the important parameters of each model's layer.…”
Federated Machine Learning (Fed ML) is a new distributed machine learning technique applied to collaboratively train a global model using clients' local data without transmitting it. Nodes only send parameter updates (e.g., weight updates in the case of neural networks), which are fused together by the server to build the global model. By not divulging node data, Fed ML guarantees its confidentiality, a crucial aspect of network security, which enables it to be used in the context of data-sensitive Internet of Things (IoT) and mobile applications, such as smart geo-location and the smart grid. However, most IoT devices are particularly energy constrained, which raises the need to optimize the Fed ML process for efficient training tasks and optimized power consumption. In this paper, we conduct, to the best of our knowledge, the first Systematic Mapping Study (SMS) on FedML optimization techniques for energy-constrained IoT devices. From a total of more than 800 papers, we select 67 that satisfy our criteria and give a structured overview of the field using a set of carefully chosen research questions. Finally, we attempt to provide an analysis of the energy-constrained Fed ML state of the art and try to outline some potential recommendations for the research community.
“…FL enables a multitude of participants to construct a joint model without sharing their private training data [4,22,23,25]. Some recent work focus on compressing the parameters or transmitting partial network for efficient transmission [2,20,28,30]. However, they do not reduce the computation overhead on edge.…”
Section: Related Workmentioning
confidence: 99%
“…Furthermore, directly training the low-rank model from scratch [12] results in the performance drop. FedDLR [28] factorizes the server model and recovers the low-rank model on the clients, reducing the communication cost while increasing the local computation cost. Pufferfish [32] improves the performance of the low-rank model by training a hybrid network and warm-up.…”
The underlying assumption of recent federated learning (FL) paradigms is that local models usually share the same network architecture as the global model, which becomes impractical for mobile and IoT devices with different setups of hardware and infrastructure. A scalable federated learning framework should address heterogeneous clients equipped with different computation and communication capabilities. To this end, this paper proposes FEDHM, a novel federated model compression framework that distributes the heterogeneous low-rank models to clients and then aggregates them into a global full-rank model. Our solution enables the training of heterogeneous local models with varying computational complexities and aggregates a single global model. Furthermore, FEDHM not only reduces the computational complexity of the device, but also reduces the communication cost by using low-rank models. Extensive experimental results demonstrate that our proposed FEDHM outperforms the current pruning-based FL approaches in terms of test Top-1 accuracy (4.6% accuracy gain on average), with smaller model size (1.5× smaller on average) under various heterogeneous FL settings.
“…In this regard, an important task in SWIPT-based FL is to optimize the portion of harvested energy allocated to communication with the edge server and local computation, respectively. In addition, the integration of SWIPT and conventional energy-saving techniques in FL, e.g., model compression, adaptive transmission, and hierarchical FL [115], needs further investigation.…”
Section: Mobile Edge Computing and Federated Learningmentioning
confidence: 99%
“…Conventional design methodologies based on mathematical optimization may not be directly applicable to large-scale SWIPT networks as the complexity of the optimal designs often scales exponentially with the network size. As a remedy, ML is a promising tool as it can be used to optimize large-scale systems without relying on analytical models [115], [116]. Recently, several papers have exploited ML for solving communication-centric resource allocation design problems in conventional wireless networks [117]- [119].…”
Section: E Machine Learning-based Designmentioning
Over the last decade, simultaneous wireless information and power transfer (SWIPT) has become a practical and promising solution for connecting and recharging batterylimited devices, thanks to significant advances in low-power electronics technology and wireless communications techniques. To realize the promised potentials, advanced resource allocation design plays a decisive role for revealing, understanding, and exploiting the intrinsic rate-energy tradeoff capitalizing on the dual use of radio frequency (RF) signals for wireless charging and communication. In this paper, we provide a comprehensive tutorial overview of SWIPT from the perspective of resource allocation design. The fundamental concepts, system architectures, and RF energy harvesting (EH) models are introduced. In particular, three commonly adopted EH models, namely the linear EH model, the nonlinear saturation EH model, and the nonlinear circuit-based EH model are characterized and discussed. Then, for a typical wireless system setup, we establish a generalized resource allocation design framework which subsumes conventional resource allocation design problems as special cases. Subsequently, we elaborate on relevant tools from optimization theory and exploit them for solving representative resource allocation design problems for SWIPT systems with and without perfect channel state information (CSI) available at the transmitter, respectively. The associated technical challenges and insights are also highlighted. Furthermore, we discuss several promising and exciting future research directions for resource allocation design for SWIPT systems intertwined with cuttingedge communication technologies, such as intelligent reflecting surfaces, unmanned aerial vehicle, mobile edge computing, federated learning, and machine learning.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.