Pose estimation for a partially observable human body from RGB-D cameras

Abstract

Human pose estimation in realistic world conditions raises multiple challenges such as foreground extraction, background update and occlusion by scene objects. Most of existing approaches were demonstrated in controlled environments. In this paper, we propose a framework to improve the performance of existing tracking methods to cope with these problems. To this end, a robust and scalable framework is provided composed of three main stages. In the first one, a probabilistic occupancy grid updated with a Hidden Markov Model used to maintain an up-to-date background and to extract moving persons. The second stage uses component labelling to identify and track persons in the scene. The last stage uses an hierarchical particle filter to estimate the body pose for each moving person. Occlusions are handled by querying the occupancy grid to identify hidden body parts so that they can be discarded from the pose estimation process. We provide a parallel implementation that runs on CPU and GPU at 4 frames per second. We also validate the approach on our own dataset that consists of synchronized motion capture with a single RGB-D camera data of a person performing actions in challenging situations with severe occlusions generated by scene objects. We make this dataset available online.

HMPO: Human Motion Prediction in Occluded Environments for Safe Motion Planning

“…This is a specialized technique for this application. From input depth images of a human occluded by obstacles [4], human joint positions can be tracked from a hierarchical particle filter, where occlusions are handled with a 3D occupancy grid and a Hidden Markov Model (HMM) is used to represent the state of visibility and occlusion. However, it is unable to track parts that are not visible.…”

“…where N is the number of human skeleton joints, h i (t) is the predicted i-th human 3D joint position at time t, and h truth,i (t) is the ground-truth human joint position. The human skeleton model-based joint tracking with particle filter [4] has an average error distance of 16.0 cm for tracking. An Extended Kalman Filter with linear motion of joint angles is used to predict the future joint positions.…”

“…The error distance of HMPO in the Occlusion dataset is 31.8 cm. HMPO reduces the error distance dataset by 38% from the particle filter-based tracking [4] plus Extended Kalman Filter (51.6 cm) and 50% from the baseline (64.0 cm). Table II highlights the performance of human action class prediction for different classification models.…”

“…Our learning algorithm classifies the human action and generates the predicted human motion using a skeleton-based human model. We add occluded images of robot scenes to existing RGBD human action prediction datasets [37,36,4]. We use these augmented datasets to train and evaluate the performance of our human motion prediction algorithm in the presence of occlusion.…”

Socially Constrained Tracking in Crowded Environments Using Shoulder Pose Estimates

“…Person detection and person tracking are both mature research areas with researchers applying sensing modalities, such as laser range finders, colour cameras, and more recently depth cameras, to detect and estimate the pose of people in a variety of scenarios. Whilst recent work on full body pose estimation using depth cameras [1], [2], [3], [4] has shown impressive results, the density of people in crowded environments and the frequency of occlusions makes reliably observing the whole body very difficult. This difficulty has caused several authors [5], [6] to focus on the parts of the body that are most visible in crowded environments, i.e.…”