Pathological Gait Classification Using Kinect v2 and Gated Recurrent Neural Networks

Jun, Kooksung; Lee, Yong-Woo; Lee, Sanghyub; Lee, Deok-Won; Kim, Mun Sang

doi:10.1109/access.2020.3013029

Cited by 34 publications

(59 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gait is an important biomedical indicator that supports a doctor or a physician in determining which body function of a patient is weakened. Therefore, many methods for analyzing human gaits by using various sensors, such as inertial sensors [1,2], planar foot pressure sensors [3][4][5][6][7], depth cameras [8][9][10][11][12][13][14][15] or motion capture systems [16,28], have been proposed. Sensor data can be used to calculate gait parameters, such as stride length, velocity, or the durations of gait phases.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning-Based Multimodal Abnormal Gait Classification Using a 3D Skeleton and Plantar Foot Pressure

Jun

Lee

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

Classification of pathological gaits has an important role in finding a weakened body part or disease and supporting a doctor's decision. Many machine learning-based approaches have been proposed that automatically classify abnormal gait patterns using various sensors, such as inertial sensors, depth cameras and foot pressure plates. In this paper, we present a deep learning-based abnormal gait classification method employing both a 3D skeleton (obtained with a depth camera) and plantar foot pressure. We collected skeleton and foot pressure data simultaneously for 1 normal and 5 pathological gaits (antalgic, lurching, steppage, stiff-legged, and Trendelenburg gaits) and classified the gaits by using single-modal models fed either skeleton or pressure data and a multimodal model fed both data types together. In the proposed method, we fed the sequential skeleton and average foot pressure data into recurrent neural network (RNN)-based encoding layers and convolutional neural network (CNN)-based encoding layers, respectively. Finally, the output features were concatenated and fed to the classification layers. The pressure-based and skeleton-based single-modal models achieved classification accuracies of 68.82% and 93.40%, respectively. The proposed multimodal hybrid model using skeleton and foot pressure data together showed improved performance, with an accuracy of 95.66%. We fine-tuned the hybrid model by applying a 3-step training methodology and ultimately increased the accuracy to 97.60%. This study indicates that the integrated features of the skeleton and foot pressure data represent both the spatiotemporal motion information and weight distribution, so data fusion can generate a positive effect in pathological gait classification.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Furthermore, a depth camera can measure the motion of the whole body, while a force pressure sensor only focuses on foot pressure. Therefore, until recently, many studies have been conducted regarding gait analysis using depth cameras [8][9][10][11][12][13][14][15].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Multimodal Abnormal Gait Classification Using a 3D Skeleton and Plantar Foot Pressure

Jun

Lee

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the context of movement-related diseases, ML/DL techniques have been used, together with data provided by wearable or vision-based sensors, to support gait assessment with the aim of diagnosis and/or evaluation of disease progression [10,11,[45][46][47][48][49][50][51][52][53][54][55][56][57]. The main focus of most contributions is the detection of abnormal gait based on information extracted from gait data obtained with accelerometers, gyroscopes and/or pressure sensors [45,46,56,57], or with RGB-D cameras [10,[47][48][49][50][51][52][53][54][55].…”

Section: Related Workmentioning

confidence: 99%

“…Beyond the classic ML algorithms, deep learning (DL) has been used -either with wearable or vision-based sensors -for gait recognition with convolutional neural networks [58] or the identification [53,54,56] or classification [52,55,57] of abnormal gait, with most using long short-term memory networks. Reported accuracy values vary between 82% [53] and 98.5% [58].…”

Section: Related Workmentioning

confidence: 99%

Supporting the Assessment of Hereditary Transthyretin Amyloidosis Patients Based On 3-D Gait Analysis and Machine Learning

Vilas-Boas

Rocha

Cardoso

et al. 2021

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Hereditary Transthyretin Amyloidosis (vATTR-V30M) is a rare and highly incapacitating sensorimotor neuropathy caused by an inherited mutation (Val30Met), which typically affects gait, among other symptoms. In this context, we investigated the possibility of using machine learning (ML) techniques to build a model(s) that can be used to support the detection of the Val30Met mutation (possibility of developing the disease), as well as symptom onset detection for the disease, given the gait characteristics of a person. These characteristics correspond to 24 gait parameters computed from 3-D body data, provided by a Kinect v2 camera, acquired from a person while walking towards the camera. To build the model(s), different ML algorithms were explored: k-nearest neighbors, decision tree, random forest, support vector machines (SVM), and multilayer perceptron. For a dataset corresponding to 66 subjects (25 healthy controls, 14 asymptomatic mutation carriers, and 27 patients) and several gait cycles per subject, we were able to obtain a model that distinguishes between controls and vATTR-V30M mutation carriers (with or without symptoms) with a mean accuracy of 92% (SVM). We also obtained a model that distinguishes between asymptomatic and symptomatic carriers with a mean accuracy of 98% (SVM). These results are very relevant, since this is the first study that proposes a ML approach to support vATTR-V30M patient assessment based on gait, being a promising foundation for the development of a computer-aided diagnosis tool to help clinicians in the identification and follow-up of this disease. Furthermore, the proposed method may also be used for other neuropathies.

show abstract

“…RNN and LSTM models have been recently applied to skeleton-based gait recognition tasks: for example, person identification [38] and pathological gait classification [39], [40]. None of these studies, however, considered the first supporting foot in the gait cycles.…”

Section: ) Recurrent Neural Network For Gait Recognitionmentioning

confidence: 99%

Physical Fatigue Detection From Gait Cycles via a Multi-Task Recurrent Neural Network

Aoki¹,

Nishikawa

Makihara

et al. 2021

IEEE Access

View full text Add to dashboard Cite

This paper describes a deep learning approach to classify physically fatigued and nonfatigued gait cycles via a recurrent neural network (RNN), where each gait cycle is represented as a time series of three-dimensional coordinates of body joints. Gait cycles inherently have large intra-class variations caused by gait stance differences (e.g., which foot is supporting/swinging) at the beginning of each gait cycle, which makes it difficult to identify subtle differences induced by fatigue. To overcome these difficulties, we introduce a supporting foot-aware RNN model in a multi-task learning framework for better fatigue detection. More specifically, the RNN model has two branches of layers: one is assigned to the main task of fatigue classification and the other is assigned to the auxiliary task of estimating the first supporting foot in the gait cycles. We collected physically fatigued and non-fatigued gait cycles from eight subjects and conducted experiments to evaluate the accuracies of the proposed multi-task model in comparison to a single-task model. As a result, the proposed method achieved an overall area under curve (AUC) of 0.860 for fatigue classification in a leave-one-subject-out cross-validation, and an AUC of 0.915 in a leave-oneday-out evaluation. It can be concluded from the experimental results that a fatigue detection system for daily use, especially for screening purposes, is very feasible on the basis of the proposed approach.INDEX TERMS Fatigue detection, gait behavior, multi-task learning, recurrent neural network, skeletal joint coordinates

show abstract

Pathological Gait Classification Using Kinect v2 and Gated Recurrent Neural Networks

Cited by 34 publications

References 37 publications

Deep Learning-Based Multimodal Abnormal Gait Classification Using a 3D Skeleton and Plantar Foot Pressure

Deep Learning-Based Multimodal Abnormal Gait Classification Using a 3D Skeleton and Plantar Foot Pressure

Supporting the Assessment of Hereditary Transthyretin Amyloidosis Patients Based On 3-D Gait Analysis and Machine Learning

Physical Fatigue Detection From Gait Cycles via a Multi-Task Recurrent Neural Network

Contact Info

Product

Resources

About