FEDZIP: A Compression Framework for Communication-Efficient Federated Learning

Malekijoo, Amirhossein; Fadaeieslam, Mohammad Javad; Malekijou, Hanieh; Homayounfar, Morteza; Alizadeh-Shabdiz, F.; Rawassizadeh, Reza

doi:10.48550/arxiv.2102.01593

Cited by 11 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The optimization of the hyper-parameters using Genetic CFL provides higher throughput for comparatively less number of rounds. Our architecture, Genetic CFL outperforms both algorithms [25] and [26] in accuracy and rounds. This holds up the fact that Genetic CFL architecture performs better while taking less number of rounds.…”

Section: Performance Analysis Of Genetic Cflmentioning

confidence: 89%

Genetic CFL: Optimization of Hyper-Parameters in Clustered Federated Learning

Agrawal,

Sarkar,

Alazab

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a distributed model for deep learning that integrates client-server architecture, edge computing, and real-time intelligence. FL has the capability of revolutionizing machine learning (ML) but lacks in the practicality of implementation due to technological limitations, communication overhead, non-IID (independent and identically distributed) data, and privacy concerns. Training a ML model over heterogeneous non-IID data highly degrades the convergence rate and performance. The existing traditional and clustered FL algorithms exhibit two main limitations, including inefficient client training and static hyper-parameter utilization. To overcome these limitations, we propose a novel hybrid algorithm, namely genetic clustered FL (Genetic CFL), that clusters edge devices based on the training hyper-parameters and genetically modifies the parameters cluster-wise. Then, we introduce an algorithm that drastically increases the individual cluster accuracy by integrating the density-based clustering and genetic hyperparameter optimization. The results are bench-marked using MNIST handwritten digit dataset and the CIFAR-10 dataset. The proposed genetic CFL shows significant improvements and works well with realistic cases of non-IID and ambiguous data.

show abstract

Section: Performance Analysis Of Genetic Cflmentioning

confidence: 89%

Genetic CFL: Optimization of Hyper-Parameters in Clustered Federated Learning

Agrawal,

Sarkar,

Alazab

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Then, they integrate additively homomorphic encryption with differential privacy to prevent data from being leaked. Malekijoo et al [75] develop a novel framework that significantly decreases the size of updates while transferring weights from the deep learning model between clients and their servers. A novel algorithm, namely FetchSGD, that compresses model updates using a Count Sketch, and takes advantage of the mergeability of sketches to combine model updates from many workers, is proposed by [76].…”

Section: Method's Category Sub-categories Studies Pros and Consmentioning

confidence: 99%

Distributed and Adaptive Edge-based AI Models for Sensor Networks (DAISeN)

Boeva¹,

Casalicchio²,

Abghari³

et al. 2022

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

This position paper describes the aims and preliminary results of the Distributed and Adaptive Edge-based AI Models for Sensor Networks (DAISeN) 1 project. The project ambition is to address today's edge AI challenges by developing advanced AI techniques that model knowledge from the sensor network and the environment to support the deployment of sustainable AI applications. We present one of the use cases being considered in DAISeN and review the state-of-the-art in three research domains related to the use case presented and directly falling into the project scope. We additionally outline the main challenges identified in each domain. The developed Global Navigation Satellite Systems (GNSS) activation model addressing the use case challenges is also briefly introduced. The future research studies planned for the remaining period of the project are finally outlined. I. INTRODUCTIONT HE NUMBER of solutions that provide Artificial Intelligence (AI) and Machine Learning (ML) based systems has been growing recently. These solutions facilitate the creation of new smart products and services in many different fields. In addition, sensor networks are undergoing great expansion and development and the integration of AI and sensor networks benefits many areas such as Industry 4.0, healthcare, mobility, logistics, and many other Internetof-Things (IoT) applications. However, this has also put new challenges in front of researchers and practitioners. New realtime AI and ML algorithms are needed along with different strategies to embed these algorithms in sensor boards and network nodes such as fog/edge nodes. For example, edgebased AI requires robust and adaptive models that take into account the temporal component of a data flow and allow for vertical and horizontal scaling of the decision-making process. These models must employ efficient learning algorithms that are capable of dealing with information varying over time and coping with large scale missing and inaccurate values. In addition, the decision-making models should be composable so that they can be distributed on the edge devices in order to This work is part of the Sony RAP 2020 Project "Distributed and Adaptive Edge-based AI Models for Sensor Networks"1 Daisen is a volcanic mountain located in Tottori Prefecture, Sanin Region of Japan. ensure a trade-off between the decision accuracy, latency, and consumed energy per decision.The IoT is an emerging key technology for future industries and the everyday lives of people, e.g., it has been playing an increasingly important role in healthcare, agriculture, home services, industrial processes, and transportation. Wireless Sensor Network (WSN) is an enabling technology for IoT [1], and, by definition, is the bridge between the physical world and the intelligence residing on the Internet. The integration of AI and sensor networks (by means of the IoT) are now realities that are changing our lives. Sensor networks are widely used to collect environmental parameters, e.g., in homes, buildings, and vehicles, where they a...

show abstract

“…In addition to ordinary data compression algorithms used to encode the final updates with lower amounts of bits, such as Huffman encoding used by [7,43], data size reduction is achieved by several other methods, such as update quantization, sparsification, and sketching. The goal of all these techniques is to reduce the amount of bits per round NTbit, sent through the wireless interface, which subsequently decreases significantly the energy usage for exchanging model's updates (equation 7).…”

Section: Updates Compressionmentioning

confidence: 99%

“…It implements a layer-wise weight quantization with an adjustable threshold during the training, which has the additional benefit of reducing the training tasks' energy budget. Similarly, [27,43,47,54] used quantization for data size reduction, in most cases mixed with other techniques. Furthermore, [27,44] proposed an adaptive schema for updating quantization to achieve communication-efficient training.…”

Section: Updates Compressionmentioning

confidence: 99%

Federated learning for energy constrained devices: a systematic mapping study

Mokadem

Maissa²,

Akkaoui³

2022

Cluster Comput

View full text Add to dashboard Cite

Federated Machine Learning (Fed ML) is a new distributed machine learning technique applied to collaboratively train a global model using clients' local data without transmitting it. Nodes only send parameter updates (e.g., weight updates in the case of neural networks), which are fused together by the server to build the global model. By not divulging node data, Fed ML guarantees its confidentiality, a crucial aspect of network security, which enables it to be used in the context of data-sensitive Internet of Things (IoT) and mobile applications, such as smart geo-location and the smart grid. However, most IoT devices are particularly energy constrained, which raises the need to optimize the Fed ML process for efficient training tasks and optimized power consumption. In this paper, we conduct, to the best of our knowledge, the first Systematic Mapping Study (SMS) on FedML optimization techniques for energy-constrained IoT devices. From a total of more than 800 papers, we select 67 that satisfy our criteria and give a structured overview of the field using a set of carefully chosen research questions. Finally, we attempt to provide an analysis of the energy-constrained Fed ML state of the art and try to outline some potential recommendations for the research community.

show abstract

FEDZIP: A Compression Framework for Communication-Efficient Federated Learning

Cited by 11 publications

References 0 publications

Genetic CFL: Optimization of Hyper-Parameters in Clustered Federated Learning

Genetic CFL: Optimization of Hyper-Parameters in Clustered Federated Learning

Distributed and Adaptive Edge-based AI Models for Sensor Networks (DAISeN)

Federated learning for energy constrained devices: a systematic mapping study

Contact Info

Product

Resources

About