Human activity recognition based on multiple inertial sensors through feature-based knowledge distillation paradigm

Mardanpour, Malihe; Sepahvand, Majid; Abdali-Mohammadi, Fardin; Nikouei, Mohammad Ali; Sarabi, Homeyra

doi:10.1016/j.ins.2023.119073

Cited by 5 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, this is the first work that exploits the relational reasoning of transformers and replaces it with 1D modalitywise convolutions to apply KD for HR estimation, which is a regression task. Other existing works on KD using wearablebased sensors, are mostly applied to classification tasks such as HAR [29], [30] and emotion recognition [31], [32], with the exception of BP waveform estimation described in [19].…”

Section: Knowledge Distillationmentioning

confidence: 99%

Replacing Attention with Modality-wise Convolution for Energy-Efficient PPG-based Heart Rate Estimation using Knowledge Distillation

Kasnesis,

Toumanidis,

Pagliari

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Continuous Hearth Rate (HR) monitoring based on photoplethysmography (PPG) sensors is a crucial feature of almost all wrist-worn devices. However, arm movements lead to the creation of Motion Artifacts (MA), affecting the performance of PPG-based HR tracking. This problem is commonly tackled by exploiting the recorded accelerometer data to correlate them with the PPG signal and eventually clean it. Thus, automatic fusion techniques based on Deep Learning (DL) algorithms have been proposed, but they are considered too large and complex to be deployed on wearable devices. The current work presents a novel and lightweight DL architecture, PULSE, comprised of temporal convolutions and feature-level multi-head cross-attention to improve sensor fusion's effectiveness. Moreover, we propose a relation-based knowledge distillation mechanism to pass PULSE's knowledge to a student network that utilizes modality-wise convolutions to replace the attention module and mimic the teacher's performance with 5x fewer parameters. The teacher and student are evaluated on the most extensive available dataset, PPG-DaLiA, with PULSE reducing the mean absolute error by 8.2% compared to the best state-of-the-art model while simultaneously reducing the inference latency by 1.6x. The student model is further compressed using post-training quantization and deployed on two microcontrollers, demonstrating its suitability for real-time execution, having a close-to-state-of-the-art MAE of 4.81 BPM (+0.40 BPM), but a 10.9x lower memory footprint of 37.9 kB, and consuming 45.9x lower energy (0.577 mJ).</p>

show abstract

Section: Knowledge Distillationmentioning

confidence: 99%

Replacing Attention with Modality-wise Convolution for Energy-Efficient PPG-based Heart Rate Estimation using Knowledge Distillation

Kasnesis,

Toumanidis,

Pagliari

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Traditional feature engineering for HAR [3] tends to be a trial-and-error process, which may vary from task to task. Hence, deep learning became popular for high-level representations of sensor-based human activities [6,7]. However, most of the deep learning based HAR rely on the supervised learning paradigm, which requires substantial labelled data for model training.…”

Section: Introductionmentioning

confidence: 99%

Wearable-based behaviour interpolation for semi-supervised human activity recognition

Duan,

Wang,

Ojha

et al. 2024

Information Sciences

View full text Add to dashboard Cite

Computer Vision Drives the New Quality Productive Forces in Agriculture: A Method for Recognizing Farming Behavior on Edge Computing Devices

Wei,

Hu,

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Human activity recognition based on multiple inertial sensors through feature-based knowledge distillation paradigm

Cited by 5 publications

References 33 publications

Replacing Attention with Modality-wise Convolution for Energy-Efficient PPG-based Heart Rate Estimation using Knowledge Distillation

Replacing Attention with Modality-wise Convolution for Energy-Efficient PPG-based Heart Rate Estimation using Knowledge Distillation

Wearable-based behaviour interpolation for semi-supervised human activity recognition

Computer Vision Drives the New Quality Productive Forces in Agriculture: A Method for Recognizing Farming Behavior on Edge Computing Devices

Contact Info

Product

Resources

About