Lifelong Adaptive Machine Learning for Sensor-Based Human Activity Recognition Using Prototypical Networks

Abstract

Continual learning (CL), also known as lifelong learning, is an emerging research topic that has been attracting increasing interest in the field of machine learning. With human activity recognition (HAR) playing a key role in enabling numerous real-world applications, an essential step towards the long-term deployment of such systems is to extend the activity model to dynamically adapt to changes in people’s everyday behavior. Current research in CL applied to the HAR domain is still under-explored with researchers exploring existing methods developed for computer vision in HAR. Moreover, analysis has so far focused on task-incremental or class-incremental learning paradigms where task boundaries are known. This impedes the applicability of such methods for real-world systems. To push this field forward, we build on recent advances in the area of continual learning and design a lifelong adaptive learning framework using Prototypical Networks, LAPNet-HAR, that processes sensor-based data streams in a task-free data-incremental fashion and mitigates catastrophic forgetting using experience replay and continual prototype adaptation. Online learning is further facilitated using contrastive loss to enforce inter-class separation. LAPNet-HAR is evaluated on five publicly available activity datasets in terms of its ability to acquire new information while preserving previous knowledge. Our extensive empirical results demonstrate the effectiveness of LAPNet-HAR in task-free CL and uncover useful insights for future challenges.

HADE: Exploiting Human Action Recognition Through Fine-Tuned Deep Learning Methods

“…Adaptive HAR systems, such as LAPNet-HAR, demonstrate a field shift towards accommodating dynamic and diverse data qualities. The LAPNet-HAR framework proposed by [46] represents adaptive learning in processing sensorbased data streams and addresses the critical issue of catastrophic forgetting in HAR. Nonetheless, as argued by [47], modeling complex human behaviors through machine learning is inherently limited by the incompressibility and unpredictability of social systems.…”

Online Continual Learning for Human Activity Recognition

“…Adaimi et al [41] argue that HAR based continual learning is still under-explored. They employ prototypical networks and memory replay to develop an online continuous learning system in which prototypes are continuously updated.…”