Semantic parametric body shape estimation from noisy depth sequences

Ichim, Alexandru Eugen; Tombari, Federico

doi:10.1016/j.robot.2015.09.029

Cited by 8 publications

(9 citation statements)

References 19 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a more recent promising work [53,54] Recently, Ichim and Tombari [55] proposed another depth camera-based algorithm for pose and shape modeling estimation. The problem is formulated as a global energy minimization problem that includes 3D correspondences (point-to-plane) and contour constrains, combined with 3D feature constraints on the joint positions (obtained as the output of NITE or Kinect2 skeleton tracking), as well as prior energy constraints to limit the tracking algorithm within a learnt/trained subspace and temporal smoothness constraints.…”

Section: Model-based Methodsmentioning

confidence: 99%

“…The method is compared against the "offline" model-based pose tracking method of [55], using the code provided by the authors at https://github.com/ aichim/bodies-ras-2015. This method takes as input a depth map sequence, as well as a noisy skeleton sequence (normally obtained as the output of a skeleton tracker, such as NITE or Kinect2 skeleton module) and simultaneously tracks the pose and estimates the human shape as a weighted average of blendshapes.…”

Section: Pose Trackingmentioning

confidence: 99%

See 1 more Smart Citation

Fast deformable model-based human performance capture and FVV using consumer-grade RGB-D sensors

Alexiadis

Zioulis

Zarpalas

et al. 2018

Pattern Recognition

View full text Add to dashboard Cite

In this paper, a novel end-to-end system for the fast reconstruction of human actor performances into 3D mesh sequences is proposed, using the input from a small set of consumer-grade RGB-Depth sensors. The proposed framework, by offline pre-reconstructing and employing a deformable actor's 3D model to constrain the on-line reconstruction process, implicitly tracks the human motion. Handling non-rigid deformation of the 3D surface and applying appropriate texture mapping, it finally produces a dynamic sequence of temporally-coherent textured meshes, enabling realistic Free Viewpoint Video (FVV). Given the noisy input from a small set of low-cost sensors, the focus is on the fast ("quick-post"), robust and fully-automatic performance reconstruction. Apart from integrating existing ideas into a complete end-to-end system, which is itself a challenging task, several novel technical advances contribute to the speed, robustness and fidelity of the system, including a layered approach for model-based pose tracking, the definition and use of sophisticated energy functions, parallelizable on the GPU, as well as

show abstract

Section: Model-based Methodsmentioning

confidence: 99%

Section: Pose Trackingmentioning

confidence: 99%

Fast deformable model-based human performance capture and FVV using consumer-grade RGB-D sensors

Alexiadis

Zioulis

Zarpalas

et al. 2018

Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…Other recent approaches to realtime body tracking use other types of capture hardware for example Kinect (RGBD) cameras [20,10] Kinect plus IMUs [8], or HTC Vive infra-red VR controllers strapped to the limbs [1].…”

Section: Related Workmentioning

confidence: 99%

“…We use two priors based on the PCA of the pose: PCA projection and PCA deviation. The projection prior encourages the solved body pose to lie close to the reduced dimensionality subspace of prior poses (soft reduction in the degrees of freedom of the joints), while the deviation prior discourages deviation from the prior observed pose variation (soft joint rotation limits) [10]. The pose projection cost is (12) and the pose deviation cost is…”

Section: Pose Prior Termsmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Full-Body Motion Capture from Video and IMUs

Malleson¹,

Gilbert

Trumble

et al. 2017

2017 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

A real-time full-body motion capture system is presented which uses input from a sparse set of inertial measurement units (IMUs) along with images from two or more standard video cameras and requires no optical markers or specialized infra-red cameras. A real-time optimization-based framework is proposed which incorporates constraints from the IMUs, cameras and a prior pose model. The combination of video and IMU data allows the full 6-DOF motion to be recovered including axial rotation of limbs and drift-free global position. The approach was tested using both indoor and outdoor captured data. The results demonstrate the effectiveness of the approach for tracking a wide range of human motion in real time in unconstrained indoor/outdoor scenes.

show abstract

Real-Time Multi-person Motion Capture from Multi-view Video and IMUs

2019

View full text Add to dashboard Cite

A real-time motion capture system is presented which uses input from multiple standard video cameras and inertial measurement units (IMUs). The system is able to track multiple people simultaneously and requires no optical markers, specialized infra-red cameras or foreground/background segmentation, making it applicable to general indoor and outdoor scenarios with dynamic backgrounds and lighting. To overcome limitations of prior video or IMU-only approaches, we propose to use flexible combinations of multiple-view, calibrated video and IMU input along with a pose prior in an online optimization-based framework, which allows the full 6-DoF motion to be recovered including axial rotation of limbs and drift-free global position. A method for sorting and assigning raw input 2D keypoint detections into corresponding subjects is presented which facilitates multi-person tracking and rejection of any bystanders in the scene. The approach is evaluated on data from several indoor and outdoor capture environments with one or more subjects and the trade-off between input sparsity and tracking performance is discussed. State-of-the-art pose estimation performance is obtained on the Total Capture (mutli-view video and IMU) and Human 3.6M (multi-view video) datasets. Finally, a live demonstrator for the approach is presented showing real-time capture, solving and character animation using a light-weight, commodity hardware setup.

show abstract

Semantic parametric body shape estimation from noisy depth sequences

Cited by 8 publications

References 19 publications

Fast deformable model-based human performance capture and FVV using consumer-grade RGB-D sensors

Fast deformable model-based human performance capture and FVV using consumer-grade RGB-D sensors

Real-Time Full-Body Motion Capture from Video and IMUs

Real-Time Multi-person Motion Capture from Multi-view Video and IMUs

Contact Info

Product

Resources

About