Resource-Efficient Continual Learning for Sensor-Based Human Activity Recognition

Abstract

Recent advances in deep learning have granted unrivaled performance to sensor-based human activity recognition (HAR). However, in a real-world scenario, the HAR solution is subject to diverse changes over time such as the need to learn new activity classes or variations in the data distribution of the already-included activities. To solve these issues, previous studies have tried to apply directly the continual learning methods borrowed from the computer vision domain, where it is vastly explored. Unfortunately, these methods either lead to surprisingly poor results or demand copious amounts of computational resources, which is infeasible for the low-cost resource-constrained devices utilized in HAR. In this paper, we provide a resource-efficient and high-performance continual learning solution for HAR. It consists of an expandable neural network trained with a replay-based method that utilizes a highly-compressed replay memory whose samples are selected to maximize data variability. Experiments with four open datasets, which were conducted on two distinct microcontrollers, show that our method is capable of achieving substantial accuracy improvements over baselines in continual learning such as Gradient Episodic Memory, while utilizing only one-third of the memory and being up to 3x faster.

A survey and perspective on neuromorphic continual learning systems

“…Some implementations use an online-unsupervised engine to generate labels for training a semi-supervised STDP-based neural network for signal processing of physiological data (Mukhopadhyay et al, 2021 ), yet the approach does not account for lost information once it retrains on new data, making adaptation slow and repetitive. Recent studies have proposed continual learning systems for wearable devices (Leite and Xiao, 2022 ). Leite and Xiao ( 2022 ) designed a dynamically expanding neural network for human activity recognition.…”

“…Recent studies have proposed continual learning systems for wearable devices (Leite and Xiao, 2022 ). Leite and Xiao ( 2022 ) designed a dynamically expanding neural network for human activity recognition. As the subjects change, the network is required to adapt to the new style without forgetting that of the previous subject.…”

Online Continual Learning for Human Activity Recognition

“…However, GANs suffer from forgetting over time and the quality of generated samples deteriorates. Leite et al [40] propose a resource-efficient strategy for HAR that utilises expandable networks that grow when new classes have to be learned. They employ the replay of compressed samples selected for maximal variability.…”