Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs

Marcard, Timo von; Rosenhahn, Bodo; Black, Michael J.; Pons-Moll, Gerard

doi:10.1111/cgf.13131

Cited by 232 publications

(202 citation statements)

References 25 publications

(34 reference statements)

Supporting

Mentioning

195

Contrasting

Unclassified

Order By: Relevance

“…For a more precise analysis we introduce additional metrics from related pose estimation areas [28,32,34]. In order to increase the robustness we furthermore suggest to i) sum until time step t rather than report the metric at time step t, ii) use more test samples covering a larger portion of the test data set and iii) evaluate the models with complementary metrics.…”

Section: Metricsmentioning

confidence: 99%

Structured Prediction Helps 3D Human Motion Modelling

Aksan

Kaufmann

Hilliges

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

188

151

View full text Add to dashboard Cite

Figure 1: We introduce a structured prediction layer (SPL) to the task of 3D human motion modelling. The SP-layer explicitly decomposes the pose into individual joints and can be interfaced with a variety of baseline architectures. We show that on H3.6M and a recent, much larger dataset, AMASS, a variety of baseline models benefit when augmented with an SP-layer. AbstractHuman motion prediction is a challenging and important task in many computer vision application domains. Existing work only implicitly models the spatial structure of the human skeleton. In this paper, we propose a novel approach that decomposes the prediction into individual joints by means of a structured prediction layer that explicitly models the joint dependencies. This is implemented via a hierarchy of small-sized neural networks connected analogously to the kinematic chains in the human body as well as a joint-wise decomposition in the loss function. The proposed layer is agnostic to the underlying network and can be used with existing architectures for motion modelling. Prior work typically leverages the H3.6M dataset. We show that some state-of-the-art techniques do not perform well when trained and tested on AMASS, a recently released dataset 14 times the size of H3.6M. Our experiments indicate that the proposed layer increases the performance of motion forecasting irrespective of the base network, jointangle representation, and prediction horizon. We furthermore show that the layer also improves motion predictions qualitatively. We make code and models publicly available at https://ait.ethz.ch/projects/2019/spl.

show abstract

Section: Metricsmentioning

confidence: 99%

Structured Prediction Helps 3D Human Motion Modelling

Aksan

Kaufmann

Hilliges

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

188

151

View full text Add to dashboard Cite

show abstract

“…By utilizing a simple game controller, Shiratori and Hodgins [22] synthesized virtual characters’ physically based locomotion sequences. Finally, von Marcard et al [2] recently proposed a method of reconstructing complex full-body motion in the wild by using six IMUs. This method has the advantage of taking into account anthropometric constraints in conjunction with a joint optimization framework to fit the model.…”

Section: Related Workmentioning

confidence: 99%

“…One research direction has aimed to reduce the number of sensors or markers required. Most motion reconstruction methods such as [1,2] use six sensors (hands, feet, head, and root) and a motion database containing sample poses of a virtual character. Thus, using the signals from the sensors, either kinematic solutions or data-driven methodologies for reconstructing full-body motion using mainly statistical motion models have been developed.…”

Section: Introductionmentioning

confidence: 99%

Full-Body Locomotion Reconstruction of Virtual Characters Using a Single Inertial Measurement Unit

Mousas

2017

Sensors

View full text Add to dashboard Cite

This paper presents a method of reconstructing full-body locomotion sequences for virtual characters in real-time, using data from a single inertial measurement unit (IMU). This process can be characterized by its difficulty because of the need to reconstruct a high number of degrees of freedom (DOFs) from a very low number of DOFs. To solve such a complex problem, the presented method is divided into several steps. The user’s full-body locomotion and the IMU’s data are recorded simultaneously. Then, the data is preprocessed in such a way that would be handled more efficiently. By developing a hierarchical multivariate hidden Markov model with reactive interpolation functionality the system learns the structure of the motion sequences. Specifically, the phases of the locomotion sequence are assigned in the higher hierarchical level, and the frame structure of the motion sequences are assigned at the lower hierarchical level. During the runtime of the method, the forward algorithm is used for reconstructing the full-body motion of a virtual character. Firstly, the method predicts the phase where the input motion belongs (higher hierarchical level). Secondly, the method predicts the closest trajectories and their progression and interpolates the most probable of them to reconstruct the virtual character’s full-body motion (lower hierarchical level). Evaluating the proposed method shows that it works on reasonable framerates and minimizes the reconstruction errors compared with previous approaches.

show abstract

“…For example, rather complex information like the shape of the skin surface or custom skeleton rig models could be included [119,182]. Hence, adding redundancies by using stochastic constraints is one possibility to address unavoidable errors in the biomechanical model parameters.…”

Section: Kinematic Calibration and Estimation Using An Optimization Amentioning

confidence: 99%

“…Explorative studies apply machine learning methods to estimate the full body postures with IMUs only attached to the extremities of the body [119,126,214].…”

Section: Increasing the Wearabilitymentioning

confidence: 99%

Assessment of hand kinematics and interactions with the environment

Kortier¹

View full text Add to dashboard Cite

The hand is one of the most important instruments of our body. Its versatility enables the execution of a wide range of tasks that ask for a powerful, precise or gentle approach. Measuring hand and finger movements, and interaction forces, is therefore important for the assessment of tasks in daily life. However, measuring on-body kinematic and kinetic quantities is a delicate procedure due to the dexterity of the hand, and moreover, the little and complex shaped skin places for sensor attachment. This thesis proposes a new on-body assessment system that allows the measurement of movements and interaction forces of the hand, fingers and thumb.The first objective, the development, evaluation and validation of an inertial and magnetic sensing system for the measurement of hand and finger kinematics is the topic of chapters 2 to 5. The second objective, assessment of the dynamic interaction between human hand and environment using combined force and movement sensing, is the topic of chapter 6.Chapter 2 describes the hardware and algorithms for a sensing system which can be attached to the hand, fingers and the thumb. The hardware consists of multiple inertial and magnetic sensors to measure angular velocities, accelerations and the magnetic field. Each individual finger and the thumb is modelled as a kinematic chain where the bones correspond to the linkages and each joint is considered as an ideal ball-socket joint. Segmental lengths were determined by manual measurement, whereas the inertial sensors provided the input for a Kalman filter to estimate the 3D orientation of the corresponding segment. Hereafter, the orientation and tip position of each finger was estimated by applying forward kinematics. To our knowledge, it is the first system that uses inertial sensors for estimating finger kinematics. The estimation quality was expressed in terms of static and dynamic accuracy, dynamic range and repeatability. Differences with an active optical reference system were found to be a maximum of 13 mm for the finger tip distance difference during circular pointing movements. A standardized test protocol for instrumented gloves showed very good repeatability results compared to other datagloves, proven by the mean angle difference of < 2 degrees. Finally, a dynamic range was specified as a measure of how well the system is able to reconstruct joint angles when experiencing large angular velocities. The system showed accurate reconstruction up to 116 full index finger flex-extension movements per minute.Chapter 3 reports an extensive comparison of our inertial sensing system against a passive opto-electronic marker system. It aims on typical hand-function tasks, including tapping, (fast) finger flexion, hand opening/closing, ab-adduction and circular pointing, which are used to quantify various motor symptoms for clinical diagnosis. Three subjects were included and instrumented with both systems. Differences in position, Range of Motion (RoM) and 3D vii joint angles were noted of which the largest were found in fast and ...

show abstract

Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs

Cited by 232 publications

References 25 publications

Structured Prediction Helps 3D Human Motion Modelling

Structured Prediction Helps 3D Human Motion Modelling

Full-Body Locomotion Reconstruction of Virtual Characters Using a Single Inertial Measurement Unit

Assessment of hand kinematics and interactions with the environment

Contact Info

Product

Resources

About