A Two-stream Convolutional Network for Musculoskeletal and Neurological Disorders Prediction

Abstract

Musculoskeletal and neurological disorders are the most common causes of walking problems among older people, and they often lead to diminished quality of life. Analyzing walking motion data manually requires trained professionals and the evaluations may not always be objective. To facilitate early diagnosis, recent deep learning-based methods have shown promising results for automated analysis, which can discover patterns that have not been found in traditional machine learning methods. We observe that existing work mostly applies deep learning on individual joint features such as the time series of joint positions. Due to the challenge of discovering inter-joint features such as the distance between feet (i.e. the stride width) from generally smaller-scale medical datasets, these methods usually perform sub-optimally. As a result, we propose a solution that explicitly takes both individual joint features and inter-joint features as input, relieving the system from the need of discovering more complicated features from small data. Due to the distinctive nature of the two types of features, we introduce a two-stream framework, with one stream learning from the time series of joint position and the other from the time series of relative joint displacement. We further develop a mid-layer fusion module to combine the discovered patterns in these two streams for diagnosis, which results in a complementary representation of the data for better prediction performance. We validate our system with a benchmark dataset of 3D skeleton motion that involves 45 patients with musculoskeletal and neurological disorders, and achieve a prediction accuracy of 95.56%, outperforming state-of-the-art methods.

Human Pose Estimation for Clinical Analysis of Gait Pathologies

“…Deep learning allows multilevel abstractions of the raw data due to its deep architecture of nonlinear hidden layers which facilitates the automatic diagnosis of neurologic damage. 9 The proposed network concatenates both LSTM and CNN for the complete extraction of spatial and temporal features of the gait cycles. The depth network contains 2 LSTM layers ( and ) followed by 3 1D convolutional layers ( , and ).…”

“…Unfortunately, large datasets are often not available in medical video/image analysis due to the restrictions on sharing data publicly in this domain. 9 Moreover, rare diseases make the process of obtaining a dataset problematic and a major impediment. So, we further train our proposed methods on various DMD videos collected from YouTube.…”

“… 5 Classify pathological gaits by inputting skeletal information and exploiting Kinect v2 and a GRU classifier which showed 90.1% performance when the entire skeleton was used and 93.67% when only the leg joints were used as input. Zhu et al 9 proposed a 2-stream CNN (2s-CNN) for the prediction of neurological disorders from small data. The 2s-CNN method involves training 2 separate neural networks, with one learning from the time series of joint position and the other from the time series of relative joint displacement.…”

“… 6 - 8 In addition, collecting, processing, and interpreting the motion data requires extensive training and experience to ensure accuracy and reliability. 9 , 10 For patients who are unable to wear such sensors due to injuries at the wrists, markerless vision-based gait tracking can create a nonintrusive means to monitor their safety. 11 Automatic diagnosis of musculoskeletal and neurological disorders using machine learning (ML) and deep neural networks (DNNs) technologies are capable of providing a cost-effective solution for the estimation of stick images or readily accessible devices (e.g.…”

“…Gait metrics, such as walking speed, stride time, cadence, symmetry, and joint angles are valuable clinical measurements that are routinely used as part of diagnostics and treatment planning for pathological gait and many other disorders, 10 including Duchenne muscular dystrophy (DMD) 12 stroke, 8 Parkinson’s disease (PD), 4 cerebral palsy, 13 multiple sclerosis, 14 Alzheimer’s disease, 15 dementia, 16 osteoarthritis, 17 and patients suffering from different functional gait disorders. 9 , 18 …”