Automated pose estimation in 3D point clouds applying annealing particle filters and inverse kinematics on a GPU

Abstract

Current experiments with HCIs have shown a high demand for more natural interaction paradigms. Gestures are thereby considered the most important cue besides speech. In order to recognize gestures it is necessary to extract meaningful motion features from the body. Up to now mostly marker based tracking systems are used in virtual reality environments, since these were traditionally more reliable than purely image based detection methods. However, markers tend to be distracting and cumbersome. Following recent advances in processing power, it becomes possible to use a camera system in order to obtain a depth image of the test subject, match it to a pre-defined body model, and thus track the body parts over time. We will present a fullbody system based on APF which enables full body tracking utilizing point clouds recorded with a 3D sensor. Further refinement is provided by a specially adapted inverse kinematics system. A GPU based implementation speeds up processing significantly and allows near real time performance.

Real-time Human Pose Estimation from Body-scanned Point Clouds

“…To do so, the depth map is used to find a similar pose within a database of prior full-body surface mesh models. Lehment et al (Lehment et al, 2010) considered 3-D point clouds extracted from depth maps to fit a mesh of a cylinder-based stickman model using Annealing Particle Filters (APF). However, the aforementioned methods require a GPU-based implementation.…”

Marker based human pose estimation using annealed particle swarm optimization with search space partitioning

“…To initialize optimization in each frame, a motion model is employed to propagate particles to the new frame. Tracking algorithms presented in [2,5,14,15] are examples of algorithms that use zero motion with additional Gaussian noise as motion model: (1) The standard deviation in the Σ matrix is equal to the maximum absolute inter-frame differences of the joint angles that are almost determined in a training process. Experiments on the motion model conducted in [6], show that tracking accuracy is largely dependent on the amount of standard deviation values in the Σ matrix.…”

Global estimation of an object’s pose using tactile sensing

“…Vision-based object tracking combined with grasp planning was first proposed by Kragic et al [1]. Yilmaz et al [2] presented a review on different tracking strategies, and the state-of-the-art in vision-based object tracking has recently seen further improvement [3][4][5]. Strategies that rely solely on vision such as [6,7], however, have limitations, particularly during manipulation tasks, where occlusions on the object are bound to occur as the robot fingers get in front of the object or it leaves the camera's field of view [8].…”