A Perceptual-Based Noise-Agnostic 3D Skeleton Motion Data Refinement Network

Li, Shujie; Zhu, Hai; Zheng, Liping; Lin, Li

doi:10.1109/access.2020.2980316

Cited by 21 publications

(29 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work by Li et al [33], [34] recognizes an aspect of the work by Holden et al [22] that we recognized in our own work, which is that much of the original natural kinematics of the data is lost when the convolutional autoencoder is used. This is because the autoencoder is trained on a rich dataset of varied Mocap data to create a manifold of human motion, but this manifold does not take into consideration the kinematics inherent amongst the sequential data.…”

Section: Deep Learning Methodsmentioning

confidence: 55%

Human Motion Enhancement via Tobit Kalman Filter-Assisted Autoencoder

2022

View full text Add to dashboard Cite

We present a novel approach to enhance the quality of human motion data collected by lowcost depth sensors, namely D-Mocap, which suffers from low accuracy and poor stability due to occlusion, interference, and algorithmic limitations. Our approach takes advantage of a large set of high-quality and diverse Mocap data by learning a general motion manifold via the convolutional autoencoder. In addition, the Tobit Kalman filter (TKF) is used to capture the kinematics of each body joint and handle censored measurement distribution. The TKF is incorporated with the autoencoder via latent space optimization, maintaining adherence to the motion manifold while preserving the kinematic nature of the original motion data. Furthermore, due to the lack of an open source benchmark dataset for this research, we have developed an extension of the Berkeley Multimodal Human Action Database (MHAD) by generating D-Mocap data from RGB-D images. The newly extended MHAD dataset is skeleton-matched and time-synced to the corresponding Mocap data and is publicly available. Along with simulated D-Mocap data generated from the CMU Mocap dataset and our self-collected D-Mocap dataset, the proposed algorithm is thoroughly evaluated and compared with different settings. Experimental results show that our approach can improve the accuracy of joint positions and angles as well as skeletal bone lengths by over 50%.INDEX TERMS Autoencoder, human motion manifold, depth sensors, motion capture, Tobit Kalman filter.

show abstract

Section: Deep Learning Methodsmentioning

confidence: 55%

Human Motion Enhancement via Tobit Kalman Filter-Assisted Autoencoder

2022

View full text Add to dashboard Cite

show abstract

“…(1) Visualization A simulation is inseparable from visualization [76,77] , and the visualization of simulation results has become a convention of a simulation. However, because the modeling of complex systems is also extremely difficult, the introduction of visualization in the early stage [78,79] not only makes the modeling of complex systems more clear and visible but also contributes to the early participation of human thinking. It can promote the correctness of modeling and the understanding of the nature of complex systems to a certain extent.…”

Section: Many Other Methods Have Also Been Proposedmentioning

confidence: 99%

State-of-the-Art Development of Complex Systems and Their Simulation Methods

Tang

Liu

2021

Complex Syst. Model. Simul.

Self Cite

View full text Add to dashboard Cite

The research on complex systems is different from that on general systems because the former must consider self-organization, emergence, uncertainty, predetermination, and evolution. As an important method to transform the world, a simulation is one of the most important skills to discover complex systems. In this study, we provide a survey on complex systems and their simulation methods. Initially, the development history of complex system research is summarized from two main lines. Then, the eight common characteristics of the most complex systems are presented. Furthermore, the simulation methods of complex systems are introduced in detail from four aspects, namely, meta-synthesis methods, complex networks, intelligent technologies, and other methods. From the overall point of view, intelligent technologies are the driving force, and complex networks are the advanced structure. Meta-synthesis methods are the integration strategy, and other methods are the supplements. In addition, we show three complex system simulation examples: digital reactor simulation, simulation of a logistics system in the industrial site, and crowd evacuation simulation. The examples show that a simulation is a useful means and an important method in complex system research.Finally, the future development prospects for complex systems and their simulation methods are suggested.

show abstract

“…The key component is the motion manifold represented in a latent space learned from the high-quality and diverse CMU Mocap dataset [21] which can be used to improve the quality of D-Mocap by projecting the erroneous or erratic data onto the latent space. In [26], researchers used a perceptual-based 3D skeleton motion data refinement network (BRA-P) to improve the refined motion data and further suppresses the bone-length variation as well as smooth the joint trajectories.…”

Section: Eai Endorsed Transactions On Bioengineering and Bioinformaticsmentioning

confidence: 99%

“…In addition, Tables 1 and 2 [6]), D-Mocap does offer advantages in low-cost and efficient motion capture. In addition, there is still much potential to be further enhanced with advanced deep learning approaches [26].…”

Section: Quantitative Evaluationmentioning

confidence: 99%

Human Motion Enhancement via Joint Optimization of Kinematic and Anthropometric Constraints

Zhou¹,

Lannan²,

Fan³

et al. 2021

EAI Endorsed Transactions on Bioengineering and Bioinformatics

View full text Add to dashboard Cite

INTRODUCTION:Recently, low cost RGB-D depth sensors have emerged as a promising alternative for motion capture for clinical gait assessment. However, depth sensor based Mocap (D-Mocap) suffers from low accuracy and poor stability for 3D joint estimation due to noise, self-occlusion, interference, and other technical limitations, which prevents it from being widely used in health-related applications. OBJECTIVES: The primary objective of this study is to integrate non-linear Kalman filters (KFs) and evolutionary algorithms to enhance the quality of D-Mocap data by jointly considering kinematic or anthropometric constraints at the joint-level and skeleton-level, respectively. METHODS: We propose a hybrid approach to synergistically integrate the Tobit Kalman filter (TKF) and the Differential Evolutionary (DE) algorithm for human motion enhancement that is referred to as TKF-DE. Specifically, the joint-level TKF provides the predictive distribution of each joint that is kinematically admissible in time and probabilistically amenable in space for skeleton-level DE optimization in terms of all bone lengths. Two predictive distributions of the TKF, i.e., the Gaussian and uniform, are tested and compared in terms of their effectiveness in generating the initial DE population. RESULTS: Two sets of motion capture data are used to validate the proposed TKF-DE methods, one simulated and one real-world. The first dataset is from the Carnegie Mellon University Database (CMU) which contains a multitude of various motions and is simplified to a 21-joint skeleton and corrupted with additive white Gaussian noise (AWGN). The second dataset was collected at two labs at Oklahoma State University (OSU). The Orbbec depth sensor and the Nuitrack SDK were used for D-Mocap data acquisition along with an optical Mocap system that was time-synchronized and skeleton-matched with the D-Mocap system as a reference for evaluation. The results confirm that the proposed TKF-DE algorithms significantly outperform other nonlinear KFs, including the extended KF (EKF), the unscented KF (UKF), and the TKF alone, with improved accuracy and stability of the estimation of joint positions, bone lengths, and joint angles. It is also shown that the Gaussian-based predictive distribution is better than the uniform one, further validating the efficacy and synergy of the two key components in the TKF-DE algorithm. CONCLUSION: Our research synergistically integrates the TKF and DE algorithms in one framework referred to as the TKF-DE, that takes advantage of kinematic and anthropometric constraints for human motion enhancement. The experimental results on two D-Mocap datasets show that the proposed TKF-DE method significantly improves the quality of D-Mocap data in terms of joint positions, bone lengths, and joint angles. This study takes one step closer to bringing the D-Mocap technology to possible health-related applications.

show abstract

A Perceptual-Based Noise-Agnostic 3D Skeleton Motion Data Refinement Network

Cited by 21 publications

References 35 publications

Human Motion Enhancement via Tobit Kalman Filter-Assisted Autoencoder

Human Motion Enhancement via Tobit Kalman Filter-Assisted Autoencoder

State-of-the-Art Development of Complex Systems and Their Simulation Methods

Human Motion Enhancement via Joint Optimization of Kinematic and Anthropometric Constraints

Contact Info

Product

Resources

About