Feature Extraction Using an RNN Autoencoder for Skeleton-Based Abnormal Gait Recognition

Jun, Kooksung; Lee, Deok-Won; Lee, Kyoobin; Lee, Sanghyub; Kim, Mun Sang

doi:10.1109/access.2020.2967845

Cited by 84 publications

(48 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RNNs are highly efficient neural networks designed for modeling sequence data such as sentences, voices, and gait patterns. RNNs are naturally more suitable for gait generation tasks than traditional feed-forward neural networks and have been widely used in gait classification [29], [30] and motion forecasting [31], [32].…”

Section: Gait Pattern Generation Modelmentioning

confidence: 99%

Individualized Gait Generation for Rehabilitation Robots Based on Recurrent Neural Networks

Zhou

Liang

Huang

et al. 2021

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Individualized reference gait patterns for lower limb rehabilitation robots can greatly improve the effectiveness of rehabilitation. However, previous methods can only generate customized gait patterns at several fixed discrete walking speeds and generating gaits at continuously varying speeds and stride lengths remains unsolved. This work proposes an individualized gait pattern generation method based on a recurrent neural network (RNN), which is proficient in series modeling. We collected the largest gait data set of this kind, which consists of 4,425 gait patterns from 137 subjects. Using this data set, we trained an RNN to create a function mapping from body parameters and gait parameters to a gait pattern. The experimental results indicate that our model is able to generate gait patterns at continuously varying walking speeds and stride lengths while also reducing the errors in the ankle, knee, and hip measurements by 12.83%, 20.95%, and 28.25%, respectively, compared to previous state-of-the-art method.

show abstract

Section: Gait Pattern Generation Modelmentioning

confidence: 99%

Individualized Gait Generation for Rehabilitation Robots Based on Recurrent Neural Networks

Zhou

Liang

Huang

et al. 2021

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

show abstract

“…This study uses a recurrent neural network (RNN) and a convolutional neural network (CNN) to process the motion data from the IMU-L sensor and the image data from the robot, respectively. RNNs are widely used for activity and gesture recognition [55][56][57], gait recognition [58][59][60], and natural language processing [61,62], and CNNs are widely used for object recognition and detection [63][64][65][66][67][68]. Because motion sensor data are sequential data, they can be analyzed with the RNN, and because an RGB image is a single-frame picture, it can be analyzed using the CNN.…”

Section: Introductionmentioning

confidence: 99%

Deep Neural Network–Based Double-Check Method for Fall Detection Using IMU-L Sensor and RGB Camera Data

et al. 2021

Self Cite

View full text Add to dashboard Cite

Existing methods for fall detection may not detect a fall when it occurs or may generate a false alarm when a fall does not occur. In order to overcome these limitations and detect falls with 100% accuracy, a double-check method for fall detection in elderly people via an inertial measurement unit-location (IMU-L) sensor and a red-green-blue (RGB) camera is proposed. The IMU-L sensor is a combination of an IMU sensor (accelerometer and gyroscope) and an ultrawideband signal-based location sensor; the RGB sensor is mounted on a robot. The proposed method involves detecting and confirming the fall of an elderly individual via the IMU-L sensor and an RGB image, respectively. The IMU-L sensor is worn on the body to detect falls. When a potential fall occurs, the individual's location information is synchronized with the motion data. During detection, because of the sequential nature of IMU data, a deep learning technique called a recurrent neural network (RNN) is trained to classify falls. When the IMU indicates a suspected fall situation, the robot moves to the corresponding location and confirms whether a fall has occurred. During the confirmation stage, a convolutional neural network-based technique is applied to the RGB image data to recognize and confirm the fall. Repeated confirmed fall detections using this method classified falls more accurately than existing methods that use only an IMU sensor. We conducted a real-time experiment to validate our method using a dataset developed in a laboratory and achieved 100% accuracy in our experimental environment. INDEX TERMS convolutional neural network, deep learning, elderly fall, fall detection, motion data with location, recurrent neural network, transfer learning.

show abstract

“…As various depth sensors and skeleton recognition techniques have been developed, many methods for skeleton-based abnormal gait recognition have been proposed [5]- [19]. However, most of these methods focused on determining whether a gait is normal or abnormal [5], [8]- [14], [19], whereas only a few studies focused on pathologically classifying gaits [6], [7], [16]- [18]. Furthermore, the target datasets were composed of only simple abnormal gaits, such as the dragging-foot gait and stiff knee gait [6], [16].…”

Section: Introductionmentioning

confidence: 99%

“…However, most of these methods focused on determining whether a gait is normal or abnormal [5], [8]- [14], [19], whereas only a few studies focused on pathologically classifying gaits [6], [7], [16]- [18]. Furthermore, the target datasets were composed of only simple abnormal gaits, such as the dragging-foot gait and stiff knee gait [6], [16]. To improve skeleton-based pathological gait classification, it is necessary to classify more complex pathological gaits.…”

Section: Introductionmentioning

confidence: 99%

“…Khokhlova et al [6] identified abnormal gaits by using a long short-term memory (LSTM)-based ensemble model and the dynamic features of the lower limbs obtained from skeleton data. Jun et al [16] extracted features from skeleton data by using a recurrent neural network (RNN)based autoencoder and fed the features to classifiers to recognize abnormal gaits. Pachón-Suescún et al [17] classified Parkinsonian, hemiplegic, and spastic diplegic gaits by using an LSTM.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pathological Gait Classification Using Kinect v2 and Gated Recurrent Neural Networks

Jun

Lee

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

With the development of depth sensors and skeleton tracking algorithms, many skeletonbased pathological gait classification methods have recently been proposed. However, these methods classify only simple gait patterns, and there is no approach to classify complicated gait patterns. In this paper, we classify 1 normal and 5 pathological gaits (antalgic, stiff-legged, lurching, steppage, and Trendelenburg gaits) by using a gated recurrent unit (GRU)-based classifier and 3D skeleton data. We collected skeleton datasets for 1 normal and 5 pathological gaits by using a multiperspective Kinect system. We developed the GRU classifier to classify the pathological gaits and compared its performance with that of other machine learning-based classifiers. Furthermore, we considered various joint groups to identify important and irrelevant joints for pathological gait classification and to improve the performance of the GRU classifier. When all skeleton data are used, the GRU classifier achieves a classification accuracy of 90.13%. A long short-term memory (LSTM)-based classifier achieves the next highest accuracy of 87.25%. The classification accuracy of the GRU classifier depends on the joint groups considered, and the classification accuracy increases to 93.67% when only leg joints are considered. This study indicates that various pathological gaits can be classified by using skeleton data and the GRU classifier. The proposed method can be used to support medical and clinical decisions. Furthermore, the results for various joint groups can be used to develop other methods in cases where only specific joint data are available because of environmental limitations.

show abstract

Feature Extraction Using an RNN Autoencoder for Skeleton-Based Abnormal Gait Recognition

Cited by 84 publications

References 38 publications

Individualized Gait Generation for Rehabilitation Robots Based on Recurrent Neural Networks

Individualized Gait Generation for Rehabilitation Robots Based on Recurrent Neural Networks

Deep Neural Network–Based Double-Check Method for Fall Detection Using IMU-L Sensor and RGB Camera Data

Pathological Gait Classification Using Kinect v2 and Gated Recurrent Neural Networks

Contact Info

Product

Resources

About