A Comparative Analysis of Hybrid Deep Learning Models for Human Activity Recognition

Abbaspour, Saedeh; Fotouhi, Faranak; Sedaghatbaf, Ali; Fotouhi, Hossein; Vahabi, Maryam; Lindén, María

doi:10.3390/s20195707

Cited by 59 publications

(29 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their CNN-LSTM model achieved the highest accuracy value with respect to the resting frameworks (93.40%); in contrast, we achieved 99% accuracy for all 4 networks tested. On the other hand, due to the imbalanced condition of our dataset, the calculated F1 score proves that the CNN-LSTM model is the best at recognition: hybrid models present higher scores [21]. Overall, using low-cost inertial sensor data with a complex Deep Learning hybrid model (CNN-LSTM) results in a good accuracy that is equal in reliability to a highercost and more complex MOCAP data system more specialized in motion capture.…”

Section: Discussionmentioning

confidence: 71%

A Deep Learning Approach to Recognize Human Activity Using Inertial Sensors and Motion Capture Systems

Jaén-Vargas

Leiva

Fernandes

et al. 2021

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Human Activity Recognition (HAR) plays an important role in behavior analysis, video surveillance, gestures recognition, gait analysis, and posture recognition. Given the recent progress of Artificial Intelligence (AI) applied to HAR, the inputs that are the data from wearable sensors can be treated as time-series from which movement events can be classified with high accuracy. In this study, a dataset of raw sensor data served as input to four different deep learning networks (DNN, CNN, LSTM, and CNN-LSTM). Differences in accuracy and learning time were then compared and evaluated for each model. An analysis of HAR was made based on an attempt to classify three activities: walking, sit-to-stand, and squatting. We also compared the performance of two different sensor data types: 3-axis linear acceleration measured from two inertial measurement units (IMUs) versus 3D acceleration of two retro-reflective markers from the high-end optoelectronic motion capture system (MOCAP). The dataset created from observations of ten subjects was preprocessed with labelling and sliding windows and then used as input to the four frameworks. The results indicate that, for HAR prediction, linear accelerations estimated using IMUs are as reliable as those measured using the MOCAP system. Also, the use of the hybrid CNN-LSTM framework for both methods resulted in higher accuracy (99%).

show abstract

Section: Discussionmentioning

confidence: 71%

A Deep Learning Approach to Recognize Human Activity Using Inertial Sensors and Motion Capture Systems

Jaén-Vargas

Leiva

Fernandes

et al. 2021

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…Each architecture has its special characteristics regarding a field of information. By merging different kinds of networks, we can extract deeper features than using the deep learning algorithm alone [ 44 ] (see Appendix C for a more detailed overview of Deep Learning). Thus, the choice of hDL architecture becomes an important point in the hDL pipeline.…”

Section: Resultsmentioning

confidence: 99%

Hybrid Deep Learning (hDL)-Based Brain-Computer Interface (BCI) Systems: A Systematic Review

et al. 2021

View full text Add to dashboard Cite

Background: Brain-Computer Interface (BCI) is becoming more reliable, thanks to the advantages of Artificial Intelligence (AI). Recently, hybrid Deep Learning (hDL), which combines different DL algorithms, has gained momentum over the past five years. In this work, we proposed a review on hDL-based BCI starting from the seminal studies in 2015. Objectives: We have reviewed 47 papers that apply hDL to the BCI system published between 2015 and 2020 extracting trends and highlighting relevant aspects to the topic. Methods: We have queried four scientific search engines (Google Scholar, PubMed, IEEE Xplore and Elsevier Science Direct) and different data items were extracted from each paper such as the database used, kind of application, online/offline training, tasks used for the BCI, pre-processing methodology adopted, type of normalization used, which kind of features were extracted, type of DL architecture used, number of layers implemented and which optimization approach were used as well. All these items were then investigated one by one to uncover trends. Results: Our investigation reveals that Electroencephalography (EEG) has been the most used technique. Interestingly, despite the lower Signal-to-Noise Ratio (SNR) of the EEG data that makes pre-processing of that data mandatory, we have found that the pre-processing has only been used in 21.28% of the cases by showing that hDL seems to be able to overcome this intrinsic drawback of the EEG data. Temporal-features seem to be the most effective with 93.94% accuracy, while spatial-temporal features are the most used with 33.33% of the cases investigated. The most used architecture has been Convolutional Neural Network-Recurrent Neural Network CNN-RNN with 47% of the cases. Moreover, half of the studies have used a low number of layers to achieve a good compromise between the complexity of the network and computational efficiency. Significance: To give useful information to the scientific community, we make our summary table of hDL-based BCI papers available and invite the community to published work to contribute to it directly. We have indicated a list of open challenges, emphasizing the need to use neuroimaging techniques other than EEG, such as functional Near-Infrared Spectroscopy (fNIRS), deeper investigate the advantages and disadvantages of using pre-processing and the relationship with the accuracy obtained. To implement new combinations of architectures, such as RNN-based and Deep Belief Network DBN-based, it is necessary to better explore the frequency and temporal-frequency features of the data at hand.

show abstract

“…Multiple authors have developed models that use a series of CNN layer first to fuse sensor data from multiple modalities before passing it to a LSTM network [33][34][35][36][37]. These achieve only minor improvements in performance classification with 95-96% accuracies.…”

Section: Related Workmentioning

confidence: 99%

Understanding LSTM Network Behaviour of IMU-Based Locomotion Mode Recognition for Applications in Prostheses and Wearables

Sherratt

Plummer

Iravani

2021

Sensors

View full text Add to dashboard Cite

Human Locomotion Mode Recognition (LMR) has the potential to be used as a control mechanism for lower-limb active prostheses. Active prostheses can assist and restore a more natural gait for amputees, but as a medical device it must minimize user risks, such as falls and trips. As such, any control system must have high accuracy and robustness, with a detailed understanding of its internal operation. Long Short-Term Memory (LSTM) machine-learning networks can perform LMR with high accuracy levels. However, the internal behavior during classification is unknown, and they struggle to generalize when presented with novel users. The target problem addressed in this paper is understanding the LSTM classification behavior for LMR. A dataset of six locomotive activities (walking, stopped, stairs and ramps) from 22 non-amputee subjects is collected, capturing both steady-state and transitions between activities in natural environments. Non-amputees are used as a substitute for amputees to provide a larger dataset. The dataset is used to analyze the internal behavior of a reduced complexity LSTM network. This analysis identifies that the model primarily classifies activity type based on data around early stance. Evaluation of generalization for unseen subjects reveals low sensitivity to hyper-parameters and over-fitting to individuals’ gait traits. Investigating the differences between individual subjects showed that gait variations between users primarily occur in early stance, potentially explaining the poor generalization. Adjustment of hyper-parameters alone could not solve this, demonstrating the need for individual personalization of models. The main achievements of the paper are (i) the better understanding of LSTM for LMR, (ii) demonstration of its low sensitivity to learning hyper-parameters when evaluating novel user generalization, and (iii) demonstration of the need for personalization of ML models to achieve acceptable accuracy.

show abstract

A Comparative Analysis of Hybrid Deep Learning Models for Human Activity Recognition

Cited by 59 publications

References 44 publications

A Deep Learning Approach to Recognize Human Activity Using Inertial Sensors and Motion Capture Systems

A Deep Learning Approach to Recognize Human Activity Using Inertial Sensors and Motion Capture Systems

Hybrid Deep Learning (hDL)-Based Brain-Computer Interface (BCI) Systems: A Systematic Review

Understanding LSTM Network Behaviour of IMU-Based Locomotion Mode Recognition for Applications in Prostheses and Wearables

Contact Info

Product

Resources

About