Human motion data refinement unitizing structural sparsity and spatial-temporal information

Wang, Zhao; Liu, Shuang; Qian, Rongqiang; Jiang, Tao; Yang, Xiaosong; Zhang, Jian Jun

doi:10.1109/icsp.2016.7877975

Cited by 8 publications

(8 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Peng et al [18] use adaptive nonnegative matrix factorization with hierarchical blocks for motion recovery. Wang et al [19] decompose the entire pose to partial models to exploit the abundant local body posture. Dictionary learning is designed and applied in parallel for each part.…”

Section: B Pose Reconstruction From Partially Observable Datamentioning

confidence: 99%

Graph-based Normalizing Flow for Human Motion Generation and Reconstruction

Yin

Kragić

et al. 2021

Preprint

View full text Add to dashboard Cite

Data-driven approaches for modeling human skeletal motion have found various applications in interactive media and social robotics. Challenges remain in these fields for generating high-fidelity samples and robustly reconstructing motion from imperfect input data, due to e.g. missed marker detection. In this paper, we propose a probabilistic generative model to synthesize and reconstruct long horizon motion sequences conditioned on past information and control signals, such as the path along which an individual is moving. Our method adapts the existing work MoGlow by introducing a new graph-based model. The model leverages the spatial-temporal graph convolutional network (ST-GCN) to effectively capture the spatial structure and temporal correlation of skeletal motion data at multiple scales. We evaluate the models on a mixture of motion capture datasets of human locomotion with foot-step and bone-length analysis. The results demonstrate the advantages of our model in reconstructing missing markers and achieving comparable results on generating realistic future poses. When the inputs are imperfect, our model shows improvements on robustness of generation.

show abstract

Section: B Pose Reconstruction From Partially Observable Datamentioning

confidence: 99%

Graph-based Normalizing Flow for Human Motion Generation and Reconstruction

Yin

Kragić

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…They usually combined PCA and Kalman smoothing together and operated models in a lower-dimensional space. Other researches [22], [29], [30], [31] introduced a chain of latent parameters to model human motion, thus a nonlinear map could be built from the latent space to the observed motion data, missing markers could be reconstructed by expectation maximum (EM) algorithm. These researches increased the accuracy of long-time missing marker reconstruction, but strong prior assumptions were imposed on these methods, leading to poor robust of irregular and complex motion.…”

Section: A Missing Marker Reconstructionmentioning

confidence: 99%

Graph Matching for Marker Labeling and Missing Marker Reconstruction With Bone Constraint by LSTM in Optical Motion Capture

Xiao

2021

IEEE Access

View full text Add to dashboard Cite

Optical motion capture (MOCAP) is a commonly used technology to record the motion of non-rigid objects with high accuracy in 3D space. However, the MOCAP data has to be processed further before it can be used. The scattered reconstructed motion data must constitute a human configuration by labelling process according to the predefined template, and the missing markers have to be reconstructed to produce a stable motion trajectory. In this work, we propose a novel labelling method for motion sequences. First, a novel graph matching method is employed to determine the connection relationship of the scattered motion data for a single frame. Then, Kalman filtering is used for tracking in the motion sequence. As for the challenge coming from missing markers, we propose a new motion data preprocessing method considering the bone length constraint, which represents the information of variation in the relative position of adjacent markers. The processed motion data is input into a Long-Short Term Memory (LSTM) model to recover the missing markers and de-noise the motion data. The experiment conducted on our own dataset proves that our labelling method achieves a similar effect to Cortex, which is a commonly used commercial motion data analysis software. The experiment on CMU dataset demonstrates that our missing marker reconstruction method can achieve an art-of-state result. The labelling code will be pulished on https://github.com/Lijianfang6930/Graph-Matching-for-Marker-Labelling INDEX TERMS Data preprocessing; Graph matching; LSTM; MOCAP data.

show abstract

“…Furthermore, the refinement of MoCap data has been addressed based on a robust matrix completion approach that takes the low-rank structure and temporal smoothness of motion data into account and is solved using the augmented Lagrange multiplier method [23]. Other approaches [24,25] perform the refinement of human motion data based on dictionary learning approaches which rely on training on high-quality data to allow appropriate results.…”

Section: Related Workmentioning

confidence: 99%

Constraint-Based Optimized Human Skeleton Extraction from Single-Depth Camera

Weinmann

et al. 2019

Sensors

View full text Add to dashboard Cite

As a cutting-edge research topic in computer vision and graphics for decades, human skeleton extraction from single-depth camera remains challenging due to possibly occurring occlusions of different body parts, huge appearance variations, and sensor noise. In this paper, we propose to incorporate human skeleton length conservation and symmetry priors as well as temporal constraints to enhance the consistency and continuity for the estimated skeleton of a moving human body. Given an initial estimation of the skeleton joint positions provided per frame by the Kinect SDK or Nuitrack SDK, which do not follow the aforementioned priors and can prone to errors, our framework improves the accuracy of these pose estimates based on the length and symmetry constraints. In addition, our method is device-independent and can be integrated into skeleton extraction SDKs for refinement, allowing the detection of outliers within the initial joint location estimates and predicting new joint location estimates following the temporal observations. The experimental results demonstrate the effectiveness and robustness of our approach in several cases.

show abstract

Human motion data refinement unitizing structural sparsity and spatial-temporal information

Cited by 8 publications

References 30 publications

Graph-based Normalizing Flow for Human Motion Generation and Reconstruction

Graph-based Normalizing Flow for Human Motion Generation and Reconstruction

Graph Matching for Marker Labeling and Missing Marker Reconstruction With Bone Constraint by LSTM in Optical Motion Capture

Constraint-Based Optimized Human Skeleton Extraction from Single-Depth Camera

Contact Info

Product

Resources

About