Estimating human 3D pose from Time-of-Flight images based on geodesic distances and optical flow

Abstract

In this paper, we present a method for human full-body pose estimation from Time-of-Flight (ToF) camera images. Our approach consists of robustly detecting anatomical landmarks in the 3D data and fitting a skeleton body model using constrained inverse kinematics. Instead of relying on appearance-based features for interest point detection that can vary strongly with illumination and pose changes, we build upon a graph-based representation of the ToF depth data that allows us to measure geodesic distances between body parts. As these distances do not change with body movement, we are able to localize anatomical landmarks independent of pose. For differentiation of body parts that occlude each other, we employ motion information, obtained from the optical flow between subsequent ToF intensity images. We provide a qualitative and quantitative evaluation of our pose tracking method on ToF sequences containing movements of varying complexity.

Multimodal speech and gesture control of AGVs, including EEG-based measurements of cognitive workload

“…A generic graph-based approach based on [8] is used to create a skeletal upper body model. In contrast to systems that use the Microsoft Kinect 1 (e.g., [9]), this approach does not rely on a large database of pre-trained body poses, which can be affected by changes in illumination, posture or camera angle.…”

Multi-touchless: Real-time fingertip detection and tracking using geodesic maxima

“…Later work by Schwarz et al [18] and Baak et al [3] built on this work proposing optimisation and recovery schemes. We employ the same idea of using geodesic extrema but in the context of finger detection and use Dijkstra's algorithm to efficiently identify these candidate regions.…”

Real-Time Range Imaging in Health Care: A Survey

“…Most systems that rely on human pose analysis exploit the mass-market proven skeletal tracking that ships with Microsoft Kinect [97]. ToF-based pose estimation has been rarely considered [98,99].…”