Replay-Driven Continual Learning for the Industrial Internet of Things

Abstract

The Industrial Internet of Things (IIoT) leverages thousands of interconnected sensors and computing devices to monitor and control large and complex industrial processes. Machine learning (ML) applications in IIoT use data acquired from multiple sensors to perform tasks such as predictive maintenance. While remembering useful learning from the past, these applications need to adapt learning for evolving sensor data stemming from changes in industrial processes and environmental conditions. This paper presents a continual learning pipeline to learn from the evolving data while replaying selected parts of the old data. The pipeline is configured to produce ML experiences (e.g., training a baseline neural network model), improve the baseline model with the new data while replaying part of the old data, and infer/predict using a specific model version given a stream of IIoT sensor data. We have evaluated our approach from an AI Engineering perspective using three industrial case studies, i.e., predicting tool wear, remaining useful lifetime, and anomalies from sensor data acquired from CNC machining and broaching operations. Our results show that configuring experiences for replay-driven continual learning allows dynamic maintenance of ML performance on evolving data while minimizing the excessive accumulation of legacy sensor data.

Uncertainty-Aware Virtual Sensors for Cyber-Physical Systems

“…Using such metrics and methods will improve the evaluation of inference scenarios and give a better picture of model reliability and the prediction process. • Continual Learning and Uncertainty Estimation: Based on the uncertainty estimation, it is possible to retrain and redeploy virtual sensors continuously in a semi-automated manner [21]. We determine the uncertainty of a virtual sensor's performance in a confidence interval for the prediction.…”