A learning-based markerless approach for full-body kinematics estimation in-natura from a single image

Abstract

We present a supervised machine learning approach for markerless estimation of human full-body kinematics for a cyclist from an unconstrained colour image. This approach is motivated by the limitations of existing marker-based approaches restricted by infrastructure, environmental conditions, and obtrusive markers. By using a discriminatively learned mixture-of-parts model, we construct a probabilistic tree representation to model the configuration and appearance of human body joints. During the learning stage, a Structured Support Vector Machine (SSVM) learns body parts appearance and spatial relations. In the testing stage, the learned models are employed to recover body pose via searching in a test image over a pyramid structure. We focus on the movement modality of cycling to demonstrate the efficacy of our approach. In natura estimation of cycling kinematics using images is challenging because of human interaction with a bicycle causing frequent occlusions. We make no assumptions in relation to the kinematic constraints of the model, nor the appearance of the scene. Our technique finds multiple quality hypotheses for the pose. We evaluate the precision of our method on two new datasets using loss functions. Our method achieves a score of 91.1 and 69.3 on mean Probability of Correct Keypoint (PCK) measure and 88.7 and 66.1 on the Average Precision of Keypoints (APK) measure for the frontal and sagittal datasets respectively. We conclude that our method opens new vistas to robust user-interaction free estimation of full body kinematics, a prerequisite to motion analysis.

A Simple Method to Optimally Select Upper-Limb Joint Angle Trajectories from Two Kinect Sensors during the Twisting Task for Posture Analysis

“…That study used a numerical mannequin with well-controlled poses to evaluate the tracking accuracy of human limb motion, with an error that increased to exceed 40° for shoulder angle measurement during a few specific body configurations when the mannequin’s body was partially occluded by other segments in the Kinect axis [ 27 ]. Another study [ 46 ] also showed that it was challenging to estimate human kinematics through Kinect sensors when occlusions were present, due to human–object interactions. By using a Kinect-based system to compute major joint angles during various tasks with/without intended occlusions, the mean error values were 13.4° and 18.3° for the tasks without intended occlusions and the tasks with intended occlusions, respectively [ 30 ].…”

Markerless 2D kinematic analysis of underwater running: A deep learning approach

“…Additional cameras were not necessary to overcome this issue, and a simple filtering procedure combined with a robust deep neural network was sufficient to produce consistent kinematic results. Nonetheless, implementing this method in 3D may help to reduce the effect of marker occlusion, due to the redundancy provided by additional cameras (see Drory et al, 2017 for a similar approach based on single images). Other difficulties included the occasional placement of a marker on the wrong limb by the neural network.…”

“…To overcome this issue, other studies have used information about spatial relations between markers (e.g. the hip is always an approximately constant distance from the knee) to better inform predictions (Drory et al, 2017), and these techniques could have helped to improve accuracy in the present study. It should also be noted that camera-based methods are not the only possible solution for kinematic analysis.…”