An On-Device Federated Learning Approach for Cooperative Model Update Between Edge Devices

Abstract

Most edge AI focuses on prediction tasks on resource-limited edge devices while the training is done at server machines. However, retraining or customizing a model is required at edge devices as the model is becoming outdated due to environmental changes over time. To follow such a concept drift, a neural-network based on-device learning approach is recently proposed, so that edge devices train incoming data at runtime to update their model. In this case, since a training is done at distributed edge devices, the issue is that only a limited amount of training data can be used for each edge device. To address this issue, one approach is a cooperative learning or federated learning, where edge devices exchange their trained results and update their model by using those collected from the other devices. In this paper, as an on-device learning algorithm, we focus on OS-ELM (Online Sequential Extreme Learning Machine) to sequentially train a model based on recent samples and combine it with autoencoder for anomaly detection. We extend it for an on-device federated learning so that edge devices can exchange their trained results and update their model by using those collected from the other edge devices. This cooperative model update is oneshot while it can be repeatedly applied to synchronize their model. Our approach is evaluated with anomaly detection tasks generated from a driving dataset of cars, a human activity dataset, and MNIST dataset. The results demonstrate that the proposed on-device federated learning can produce a merged model by integrating trained results from multiple edge devices as accurately as traditional backpropagation based neural networks and a traditional federated learning approach with lower computation or communication cost.

Machine Learning for the Control and Monitoring of Electric Machine Drives: Advances and Trends

“…Furthermore, as is common with FPGAs in general, they are mostly limited to the inference of a neural network, and their use for online learning through backpropagation is often difficult in general due to the limited resources available. Nevertheless, some on-device learning approaches that do not rely on backpropagation for training have been proposed for FPGAs, such as those discussed in [253], [254], and [255].…”

Fast Deep Autoencoder for Federated learning

“…For anomaly detection in edge computing and federated learning scenarios, R. Ito et al [19] propose to combine OS-ELM (Online Sequential Extreme Learning Machine) [20] with autoencoders. This allows each edge device to train its own local model and incrementally update it with the results obtained by the other devices.…”

Federated Learning for Internet of Things

“…In order to verify the effectiveness of FL for IoT security, different from previous works [2,4,8,12,15], we hope to learn a detection model from benign data widely distributed in different types of devices that can identify a larger range of attacks. Specifically, we hope that the data design meets three characteristics: 1.…”

“…IoTDefender [2] is another similar framework but obtains a personalized model by fine-tuning the global model trained with federated learning. [4] evaluates FL-based anomaly detection framework with learning tasks such as aggressive driving detection and human activity recognition. [8] further proposed an attentionbased CNN-LSTM model to detect anomalies in an FL manner, and reduced the communication cost by using Top-𝑘 gradient compression.…”