Deep Learning for Sensor-Based Rehabilitation Exercise Recognition and Evaluation

Abstract

In this paper, a multipath convolutional neural network (MP-CNN) is proposed for rehabilitation exercise recognition using sensor data. It consists of two novel components: a dynamic convolutional neural network (D-CNN) and a state transition probability CNN (S-CNN). In the D-CNN, Gaussian mixture models (GMMs) are exploited to capture the distribution of sensor data for the body movements of the physical rehabilitation exercises. Then, the input signals and the GMMs are screened into different segments. These form multiple paths in the CNN. The S-CNN uses a modified Lempel–Ziv–Welch (LZW) algorithm to extract the transition probabilities of hidden states as discriminate features of different movements. Then, the D-CNN and the S-CNN are combined to build the MP-CNN. To evaluate the rehabilitation exercise, a special evaluation matrix is proposed along with the deep learning classifier to learn the general feature representation for each class of rehabilitation exercise at different levels. Then, for any rehabilitation exercise, it can be classified by the deep learning model and compared to the learned best features. The distance to the best feature is used as the score for the evaluation. We demonstrate our method with our collected dataset and several activity recognition datasets. The classification results are superior when compared to those obtained using other deep learning models, and the evaluation scores are effective for practical applications.

Recognition and Repetition Counting for Local Muscular Endurance Exercises in Exercise-Based Rehabilitation: A Comparative Study Using Artificial Intelligence Models

“…This approach of multipath CNN-based learning is a combination of feature learning from different methods and is different to our approach as we used a single deep CNN model for the exercise recognition and repetition counting. However, our study also differs with [ 35 ] in that we employed a greater number of exercises, more diverse limb movements and larger limb movements in the exercises.…”

“…A third study, by Zheng-An et al [ 35 ], used a multipath CNN model for sensor-based rehabilitation exercise recognition. The study made use of a CNN model based on Gaussian mixer models on the wearable sensor data as first channel path information and second CNN to calculate state transition probability using Lemple–Ziv–Welch coding.…”

“…While our study and those of Soro et al [ 27 ], Um Terry et al [ 23 ] and Zheng-An et al [ 35 ] used different exercises and different data-sets, they all have tried to address exercise-based rehabilitation using deep learning models. The present state-of-the-art deep CNNs appear to show higher accuracy measures in comparison to the supervised ML models due to the ability of deep CNNs to learn a higher number of features in comparison to the associated handcrafted feature learning with ML models.…”

“…In recent years, many ML and deep learning-based models have been used along with wearable sensors in the assessment of human movement activities in many domains including: health [ 11 ], recreation activities [ 12 ], musculoskeletal injuries or diseases [ 13 ], day-to-day routine activities (e.g., walking, jogging, running, sitting, drinking, watching TV) [ 11 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ], sporting movements [ 22 ] and exercises [ 23 , 24 , 25 , 26 , 27 ]. The ML models used for exercise recognition have predominantly used multiple wearable sensors [ 28 , 29 , 30 , 31 ], specifically in the areas of free weight exercise monitoring [ 32 ], the performance of lunge evaluation [ 24 ], limb movement rehabilitation [ 33 ], intensity recognition in strength training [ 34 ], exercise feedback [ 24 ], qualitative evaluation of human movements [ 28 ], gym activity monitoring [ 29 ], rehabilitation [ 23 , 25 , 33 , 35 ] and indoor-based exercises for strength training [ 36 ]. However, the use of multiple sensors is far from ideal in practice because of cost, negative aesthetics and reduced user uptake [ 17 ].…”

Rehabilitation Exercise Recognition and Evaluation Based on Smart Sensors With Deep Learning Framework

“…Zhu et al [24] initialized the Multi-path CNN for the recognition of rehabilitation exercise. Results of classification accuracy demonstrated by the experiments that an MP-CNN is highly efficient for sensor data acquisition.…”