MAG-Res2Net: a novel deep learning network for human activity recognition

Abstract

Objective. Human activity recognition (HAR) has become increasingly important in healthcare, sports, and fitness domains due to its wide range of applications. However, existing deep learning based HAR methods often overlook the challenges posed by the diversity of human activities and data quality, which can make feature extraction difficult. To address these issues, we propose a new neural network model called MAG-Res2Net, which incorporates the Borderline-SMOTE data upsampling algorithm, a loss function combination algorithm based on metric learning, and the Lion optimization algorithm. Approach. We evaluated the proposed method on two commonly utilized public datasets, UCI-HAR and WISDM, and leveraged the CSL-SHARE multimodal human activity recognition dataset for comparison with state-of-the-art models. Main results. On the UCI-HAR dataset, our model achieved accuracy, F1-macro, and F1-weighted scores of 94.44%, 94.38%, and 94.26%, respectively. On the WISDM dataset, the corresponding scores were 98.32%, 97.26%, and 98.42%, respectively. Significance. The proposed MAG-Res2Net model demonstrates robust multimodal performance, with each module successfully enhancing model capabilities. Additionally, our model surpasses current human activity recognition neural networks on both evaluation metrics and training efficiency. Source code of this work is available at: https://github.com/LHY1007/MAG-Res2Net.

IMU-Based Fitness Activity Recognition Using CNNs for Time Series Classification

“…Recently, modified and extended ResNet architectures specifically aimed at human activity recognition were proposed, including MAG-Res2Net [5] and an architecture [6] based on ResNeXt [36]. Whereas these architectures have shown promising results on the HAR data sets on which they were tested, the architecture proposed by Mekruksavanich et al [6] does not compare ResNeXt with ResNet, and both publications were not available at the time of our study, leading us to use the regular ResNet architecture in this work.…”

“…Although most commonly used for visual tasks, convolutional neural networks (CNNs) have been shown to produce competitive results on time series classification tasks [3], including human activity recognition (HAR) [4][5][6]. In the context of fitness activity recognition, they have been successfully applied to various different activities, such as swing sports [7][8][9], skiing [10,11], beach volleyball [12], football [13], and exercising [14].…”

Deep Wavelet Convolutional Neural Networks for Multimodal Human Activity Recognition Using Wearable Inertial Sensors

“…In the time domain, current research primarily focuses on sensor signal-based HAR using DL techniques. These techniques include CNN [ 20 , 23 , 26 , 29 , 40 ], variants of RNNs like LSTM and GRU, and hybrid DL methods [ 19 , 27 , 28 , 41 , 42 , 43 ]. Authors such as Yan et al [ 40 ], Cheng et al [ 26 ], and Wang et al [ 23 ] have introduced supporting techniques like the attention layer and convolution layers with various kernel sizes to enhance CNNs for HAR, thereby modifying the original CNN model architectures.…”

“…Authors such as Yan et al [ 40 ], Cheng et al [ 26 ], and Wang et al [ 23 ] have introduced supporting techniques like the attention layer and convolution layers with various kernel sizes to enhance CNNs for HAR, thereby modifying the original CNN model architectures. Liu et al [ 29 ] proposed MAG-Res2Net, which explored two DL architectures ResNet [ 30 ] and added the gated module to improve performance multimodal HAR on three simple and more complex public datasets such as UCI-HAR [ 31 ] and WISDM [ 32 ] and leveraged the CSL-SHARE dataset [ 33 ]. MAG-Res2Net, belonging to the time domain, utilizes raw signals directly without undergoing signal transformation via wavelet transformation.…”

“…Specifically, convolutional neural networks (CNN) [ 20 , 23 , 24 , 25 , 26 ], hybrid CNN and long short-term memory (LSTM) [ 19 , 27 , 28 ] are utilized to automatically extract deep features in HAR systems without relying on human domain knowledge. MAG-Res2Net [ 29 ] is the latest publication method, which proposed two DL architectures of ResNet [ 30 ] with adding a gating module. The MAG-Res2Net model demonstrated robust multimodal performance on two commonly public datasets UCI-HAR [ 31 ] and WISDM [ 32 ] and leveraged the CSL-SHARE dataset [ 33 ].…”