A CNN-LSTM Approach to Human Activity Recognition

Mutegeki, Ronald; Han, Dong Seog

doi:10.1109/icaiic48513.2020.9065078

Cited by 227 publications

(62 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reference [20] used a threelayer LSTM model. Although reference [28] has an accuracy similar to ours, like references [33] and [34] it did not use only CNN but also LSTM, and thus the complexity of the algorithm was increased. This study used only CNN and could obtain similar or even superior accuracy.…”

Section: H the Comparisons Of Several Models Of The Open Datasetmentioning

confidence: 89%

See 1 more Smart Citation

Human Daily Activity Recognition Performed Using Wearable Inertial Sensors Combined With Deep Learning Algorithms

2020

View full text Add to dashboard Cite

This study proposed a wearable device capable of recognizing six human daily activities (walking, walking upstairs, walking downstairs, sitting, standing, and lying) through a deep learning algorithm. Existing wearable devices are mainly watches or wristbands, and almost none are to be worn on the waist. Wearable devices in the forms of watches and wristbands are unfriendly to patients who are critically ill, such as patients undergoing dialysis. Patients undergoing dialysis have artificial blood vessels on their arm, and they cannot perform intense exercise. For this type of users, general hand wearable devices cannot correctly identify wearers' activities. Therefore, we proposed a waist wearable device and these types of daily life activities to assess their exercise. The hardware of the wearable device consisted of an inertial sensor, which included a microcontroller, a three-axis accelerometer, and a three-axis gyroscope. The activity recognition algorithm of the software used motion signals acquisition, signal normalization, and a feature learning method. The feature learning method was based on a 1D convolutional neural network that automatically performed feature extraction and classification from raw data. One part of the experimental data was from the dataset of the University of California (UCI), and the other part was recorded by this study. To capture the data recorded, the wearable inertial sensing device was attached to the waists of 21 experimental participants who performed six common movements in a laboratorial environment, and the subsequent records were collected to verify the validity of the proposed deep learning algorithm in relation to the inertial sensor of the wearable device. For the six common activities in the UCI dataset and the data recorded, the recognition rates in the training sample reached 98.93% and 97.19%, respectively, and the recognition rates in the testing sample were 95.99% and 93.77%, respectively.

show abstract

Section: H the Comparisons Of Several Models Of The Open Datasetmentioning

confidence: 89%

“…After 10 repetitions, the average accuracy of the open database was 95.08%, and the mean accuracy of the database we recorded was 87.88%. [20] 93.70% LSTM-CNN [28] 95.78% Bidir-LSTM [31] 93.79% EHARS [32] 93.92% CNN-LSTM [33] 92.13% CNN-LSTM [34] 93.40% Ours 95.99%…”

Section: G K-fold Cross-validation In Both Open Dataset and Data Thimentioning

confidence: 99%

Human Daily Activity Recognition Performed Using Wearable Inertial Sensors Combined With Deep Learning Algorithms

2020

View full text Add to dashboard Cite

show abstract

“…Multiple authors have developed models that use a series of CNN layer first to fuse sensor data from multiple modalities before passing it to a LSTM network [33][34][35][36][37]. These achieve only minor improvements in performance classification with 95-96% accuracies.…”

Section: Related Workmentioning

confidence: 99%

Understanding LSTM Network Behaviour of IMU-Based Locomotion Mode Recognition for Applications in Prostheses and Wearables

Sherratt

Plummer

Iravani

2021

Sensors

View full text Add to dashboard Cite

Human Locomotion Mode Recognition (LMR) has the potential to be used as a control mechanism for lower-limb active prostheses. Active prostheses can assist and restore a more natural gait for amputees, but as a medical device it must minimize user risks, such as falls and trips. As such, any control system must have high accuracy and robustness, with a detailed understanding of its internal operation. Long Short-Term Memory (LSTM) machine-learning networks can perform LMR with high accuracy levels. However, the internal behavior during classification is unknown, and they struggle to generalize when presented with novel users. The target problem addressed in this paper is understanding the LSTM classification behavior for LMR. A dataset of six locomotive activities (walking, stopped, stairs and ramps) from 22 non-amputee subjects is collected, capturing both steady-state and transitions between activities in natural environments. Non-amputees are used as a substitute for amputees to provide a larger dataset. The dataset is used to analyze the internal behavior of a reduced complexity LSTM network. This analysis identifies that the model primarily classifies activity type based on data around early stance. Evaluation of generalization for unseen subjects reveals low sensitivity to hyper-parameters and over-fitting to individuals’ gait traits. Investigating the differences between individual subjects showed that gait variations between users primarily occur in early stance, potentially explaining the poor generalization. Adjustment of hyper-parameters alone could not solve this, demonstrating the need for individual personalization of models. The main achievements of the paper are (i) the better understanding of LSTM for LMR, (ii) demonstration of its low sensitivity to learning hyper-parameters when evaluating novel user generalization, and (iii) demonstration of the need for personalization of ML models to achieve acceptable accuracy.

show abstract

“…The findings revealed that with no extra difficulty in training, the Stacked LSTM network could enhance recognition accuracy. Better recognition performance was achieved by combining the CNN network with the LSTM network, based on the study by Mutegeki et al [ 31 ] who used the robustness of CNN network feature extraction while taking advantage of the LSTM model for the classification of time series. To provide promising results in recognition performance, Ordóñez and Roggen [ 32 ] combined the convolutional layer with LSTM layers.…”

Section: Introductionmentioning

confidence: 99%

LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes

Mekruksavanich

Jitpattanakul

2021

Sensors

170

View full text Add to dashboard Cite

Human Activity Recognition (HAR) employing inertial motion data has gained considerable momentum in recent years, both in research and industrial applications. From the abstract perspective, this has been driven by an acceleration in the building of intelligent and smart environments and systems that cover all aspects of human life including healthcare, sports, manufacturing, commerce, etc. Such environments and systems necessitate and subsume activity recognition, aimed at recognizing the actions, characteristics, and goals of one or more individuals from a temporal series of observations streamed from one or more sensors. Due to the reliance of conventional Machine Learning (ML) techniques on handcrafted features in the extraction process, current research suggests that deep-learning approaches are more applicable to automated feature extraction from raw sensor data. In this work, the generic HAR framework for smartphone sensor data is proposed, based on Long Short-Term Memory (LSTM) networks for time-series domains. Four baseline LSTM networks are comparatively studied to analyze the impact of using different kinds of smartphone sensor data. In addition, a hybrid LSTM network called 4-layer CNN-LSTM is proposed to improve recognition performance. The HAR method is evaluated on a public smartphone-based dataset of UCI-HAR through various combinations of sample generation processes (OW and NOW) and validation protocols (10-fold and LOSO cross validation). Moreover, Bayesian optimization techniques are used in this study since they are advantageous for tuning the hyperparameters of each LSTM network. The experimental results indicate that the proposed 4-layer CNN-LSTM network performs well in activity recognition, enhancing the average accuracy by up to 2.24% compared to prior state-of-the-art approaches.

show abstract

A CNN-LSTM Approach to Human Activity Recognition

Cited by 227 publications

References 3 publications

Human Daily Activity Recognition Performed Using Wearable Inertial Sensors Combined With Deep Learning Algorithms

Human Daily Activity Recognition Performed Using Wearable Inertial Sensors Combined With Deep Learning Algorithms

Understanding LSTM Network Behaviour of IMU-Based Locomotion Mode Recognition for Applications in Prostheses and Wearables

LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes

Contact Info

Product

Resources

About