3D Tracking of Human Motion Using Visual Skeletonization and Stereoscopic Vision

Abstract

The design of markerless systems to reconstruct human motion in a timely, unobtrusive and externally valid manner is still an open challenge. Artificial intelligence algorithms based on automatic landmarks identification on video images opened to a new approach, potentially e-viable with low-cost hardware. OpenPose is a library that t using a two-branch convolutional neural network allows for the recognition of skeletons in the scene. Although OpenPose-based solutions are spreading, their metrological performances relative to video setup are still largely unexplored. This paper aimed at validating a two-cameras OpenPose-based markerless system for gait analysis, considering its accuracy relative to three factors: cameras' relative distance, gait direction and video resolution. Two volunteers performed a walking test within a gait analysis laboratory. A marker-based optical motion capture system was taken as a reference. Procedures involved: calibration of the stereoscopic system; acquisition of video recordings, simultaneously with the reference marker-based system; video processing within OpenPose to extract the subject's skeleton; videos synchronization; triangulation of the skeletons in the two videos to obtain the 3D coordinates of the joints. Two set of parameters were considered for the accuracy assessment: errors in trajectory reconstruction and error in selected gait space-temporal parameters (step length, swing and stance time). The lowest error in trajectories (∼20 mm) was obtained with cameras 1.8 m apart, highest resolution and straight gait, and the highest (∼60 mm) with the 1.0 m, low resolution and diagonal gait configuration. The OpenPose-based system tended to underestimate step length of about 1.5 cm, while no systematic biases were found for swing/stance time. Step length significantly changed according to gait direction (p = 0.008), camera distance (p = 0.020), and resolution (p < 0.001). Among stance and swing times, the lowest errors (0.02 and 0.05 s for stance and swing, respectively) were obtained with the 1 m, highest resolution and straight gait configuration. These findings confirm the feasibility of tracking kinematics and gait parameters of a single subject in a 3D space using two low-cost webcams and the OpenPose engine. In particular, the maximization of cameras distance and video resolution enabled to achieve the highest metrological performances.

Adaptive Rehabilitation Bots in Serious Games

“…Kinect 2 is well known for its ability to track 25 joints simultaneously allowing it to capture the full human body gestures. However, Kinect 2 sometimes falls short when it comes to (1) capturing subtle hand motions with high accuracy, (2) recognizing short-range motions, (3) functioning in bright sun light, and (4) immunity to interference caused by other nearby sensors [ 26 ]. A recent work [ 27 ] demonstrates a superiority in pose tracking performance of the Azure Kinect over the Kinect 2, in which the Azure Kinect more accurately measured the spatial gate parameters.…”

Spatiotemporal Gait Measurement With a Side-View Depth Sensor Using Human Joint Proposals

“…Markerless gait analysis has been performed using RGB cameras, depth sensors, and other devices such as laser scanners. Recently, Zago et al [15] measured spatiotemporal parameters without markers using two RGB cameras for stereoscopic vision. Iwai et al [16] used 2D laser range sensors placed at shin height to estimate where the foot contacts the ground, but required subjects to walk barefoot with shins exposed (e.g., wearing shorts).…”

Video-based quantification of human movement frequency using pose estimation

“…In this study, we used OpenPose (a freely available human pose estimation algorithm (11,15,16) to measure the frequencies of repetitive upper and lower extremity movements using smartphone videos. Previous studies have used OpenPose to study human gait (17)(18)(19)(20)(21) and quantify tremor severity in persons with PD (22). Other pose estimation algorithms have been used to measure the degree of levodopa-induced dyskinesias (23) and finger tapping frequency in PD that correlated well with clinical ratings (24,25).…”