Predicting TUG score from gait characteristics with video analysis and machine learning

Abstract

Fall is a leading cause of death which suffers the elderly and society. Timed Up and Go (TUG) test is a common tool for fall risk assessment. In this paper, we propose a method for predicting TUG score from gait characteristics extracted from video with computer vision and machine learning technologies. First, 3D pose is estimated from video captured with 2D and 3D cameras during human motion and then a group of gait characteristics are computed from 3D pose series. After that, copula entropy is used to select those characteristics which are mostly associated with TUG score. Finally, the selected characteristics are fed into the predictive models to predict TUG score. Experiments on real world data demonstrated the effectiveness of the proposed method. As a byproduct, the associations between TUG score and several gait characteristics are discovered, which laid the scientific foundation of the proposed method and make the predictive models such built interpretable to clinical users.

Estimating Postural Stability Using Improved Permutation Entropy via TUG Accelerometer Data for Community-Dwelling Elderly People

“…Specifically, they found that all are distinguishable in TUG, the Berg balance scale (BBS), and the combined TUG and the short-form Berg balance scale (SFBBS), indicating that MSE can effectively classify the participants of these clinical tests using behavioral actions. Jian et al (2020) [ 17 ] also employed entropy with video analysis and machine learning to predict the TUG score from gait characteristics. Their results showed that copula entropy is used to select the characteristics that are mainly related to the TUG score.…”

Deep Learning-Based Subtask Segmentation of Timed Up-and-Go Test Using RGB-D Cameras

“…Because the TUG test consists of relatively simple activities, it can be easily distinguished using low-dimensional features, such as human posture detected by body keypoints [18]. Therefore, previous studies investigated various types of input data to obtain efficient results with subtask segmentation [3,4,18,22,23,28,35,37,41] or fall risk prediction [25,42]. We summarize these in Table A1 in the Appendix A.…”

“…These findings motivated us to speculate that similar locations might be better than other locations in the TUG subtask segmentation task by using a lower resolution of RGB-D than the IMU. Previous studies using RGB or RGB-D cameras used complete skeleton data spread throughout the body as inputs to ANN models or rules [18,22,42]. We hypothesized that the input(s) only from the proximal joints, for example, center of gravity (COG) movement or head motion, and not with all 32 skeleton joints, should be sufficient for classifying TUG subtasks.…”