Train++: An Incremental ML Model Training Algorithm to Create Self-Learning IoT Devices

Sudharsan, Bharath; Yadav, Piyush; Breslin, John G.; Ali, Muhammad Intizar

doi:10.1109/swc50871.2021.00023

Cited by 19 publications

(17 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their approach allows for improving the running model at run-time without relying on cloud entities. Instead, in another of their papers [ 35 ], the authors present Train++, an algorithm to locally and incrementally train a TinyML model directly on an embedded device without relying on a cloud or any other external service. The target ML problem has to be reduced to a set of binary classifiers.…”

Section: Related Workmentioning

confidence: 99%

An Adaptable and Unsupervised TinyML Anomaly Detection System for Extreme Industrial Environments

Antonini

Pincheira

Vecchio

et al. 2023

Sensors

View full text Add to dashboard Cite

Industrial assets often feature multiple sensing devices to keep track of their status by monitoring certain physical parameters. These readings can be analyzed with machine learning (ML) tools to identify potential failures through anomaly detection, allowing operators to take appropriate corrective actions. Typically, these analyses are conducted on servers located in data centers or the cloud. However, this approach increases system complexity and is susceptible to failure in cases where connectivity is unavailable. Furthermore, this communication restriction limits the approach’s applicability in extreme industrial environments where operating conditions affect communication and access to the system. This paper proposes and evaluates an end-to-end adaptable and configurable anomaly detection system that uses the Internet of Things (IoT), edge computing, and Tiny-MLOps methodologies in an extreme industrial environment such as submersible pumps. The system runs on an IoT sensing Kit, based on an ESP32 microcontroller and MicroPython firmware, located near the data source. The processing pipeline on the sensing device collects data, trains an anomaly detection model, and alerts an external gateway in the event of an anomaly. The anomaly detection model uses the isolation forest algorithm, which can be trained on the microcontroller in just 1.2 to 6.4 s and detect an anomaly in less than 16 milliseconds with an ensemble of 50 trees and 80 KB of RAM. Additionally, the system employs blockchain technology to provide a transparent and irrefutable repository of anomalies.

show abstract

Section: Related Workmentioning

confidence: 99%

An Adaptable and Unsupervised TinyML Anomaly Detection System for Extreme Industrial Environments

Antonini

Pincheira

Vecchio

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…First, the training framework must be able to detect when a significant shift has happened in the input dataset (when to learn). This can be done by calculating the per-output covariate distribution divergence on principal feature components [183], running mean and variance of streaming input [184], or confidence score of predictions [186]. Second, the on-device training framework must perform model adaptation within device constraints and limited training samples (how to learn).…”

Section: A On-device Trainingmentioning

confidence: 99%

“…This allows learning from non-IID data. Incremental training uses constrained optimization to update the weights one sample at a time [186]. Both approaches suffer from limited application space due to limited supported network types.…”

Section: A On-device Trainingmentioning

confidence: 99%

Machine Learning for Microcontroller-Class Hardware: A Review

2022

View full text Add to dashboard Cite

The advancements in machine learning (ML) opened a new opportunity to bring intelligence to the low-end Internet-of-Things (IoT) nodes, such as microcontrollers. Conventional ML deployment has high memory and computes footprint hindering their direct deployment on ultraresource-constrained microcontrollers. This article highlights the unique requirements of enabling onboard ML for microcontroller-class devices. Researchers use a specialized model development workflow for resource-limited applications to ensure that the compute and latency budget is within the device limits while still maintaining the desired performance. We characterize a closed-loop widely applicable workflow of ML model development for microcontroller-class devices and show that several classes of applications adopt a specific instance of it. We present both qualitative and numerical insights into different stages of model development by showcasing several use cases. Finally, we identify the open research challenges and unsolved questions demanding careful considerations moving forward.

show abstract

“…CL has already been explored for classic server and parallel architectures, but only in the last period focused on resource-constrained platforms. As of now, there already exist some well performing strategies and frameworks like TinyTL [13], Progress & Compress [14], TinyOL [15], and Train++ [16]. CL has already been successfully applied in a domain of interest for our study: image classification.…”

Section: Related Workmentioning

confidence: 99%

Incremental Online Learning Algorithms Comparison for Gesture and Visual Smart Sensors

Avi

Albanese

Brunelli

2022

2022 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Tiny machine learning (TinyML) in IoT systems exploits MCUs as edge devices for data processing. However, traditional TinyML methods can only perform inference, limited to static environments or classes. Real case scenarios usually work in dynamic environments, thus drifting the context where the original neural model is no more suitable. For this reason, pre-trained models reduce accuracy and reliability during their lifetime because the data recorded slowly becomes obsolete or new patterns appear. Continual learning strategies maintain the model up to date, with runtime fine-tuning of the parameters. This paper compares four state-of-the-art algorithms in two real applications: i) gesture recognition based on accelerometer data and ii) image classification. Our results confirm these systems' reliability and the feasibility of deploying them in tiny-memory MCUs, with a drop in the accuracy of a few percentage points with respect to the original models for unconstrained computing platforms.

show abstract

Train++: An Incremental ML Model Training Algorithm to Create Self-Learning IoT Devices

Cited by 19 publications

References 19 publications

An Adaptable and Unsupervised TinyML Anomaly Detection System for Extreme Industrial Environments

An Adaptable and Unsupervised TinyML Anomaly Detection System for Extreme Industrial Environments

Machine Learning for Microcontroller-Class Hardware: A Review

Incremental Online Learning Algorithms Comparison for Gesture and Visual Smart Sensors

Contact Info

Product

Resources

About