Magnetometer Robust Deep Human Pose Regression With Uncertainty Prediction Using Sparse Body Worn Magnetic Inertial Measurement Units

Abstract

We propose a deep learning based framework that learns data-driven temporal priors to perform 3D human pose estimation from six body worn Magnetic Inertial Measurement units sensors. Our work estimates 3D human pose with associated uncertainty from sparse body worn sensors. We derive and implement a 3D angle representation that eliminates yaw angle (or magnetometer dependence) and show that 3D human pose is still obtained from this reduced representation, but with enhanced uncertainty. We do not use kinematic acceleration as input and show that it improves the generalization to real sensor data from different subjects as well as accuracy. Our framework is based on Bi-directional recurrent autoencoder. A sliding window is used at inference time, instead of full sequence (offline mode). The major contribution of our research is that 3D human pose is predicted from sparse sensors with a well calibrated uncertainty which is correlated with ambiguity and actual errors. We have demonstrated our results on two real sensor datasets; DIP-IMU and Total capture and have come up with state-of-art accuracy. Our work confirms that the main limitation of sparse sensor based 3D human pose prediction is the lack of temporal priors. Therefore fine-tuning on a small synthetic training set of target domain, improves the accuracy.

Uncertainty prediction in human pose estimation based on Gaussian heatmap

“…Recently, uncertainty prediction has become an important method for many tasks [42], [43], [44], [45] Uncertainty prediction can be summarized epistemic and aleatoric uncertainty [46]. The former is caused by the lack of understanding of the data distribution and the latter is related to the noise inherent in the process of generating the data, such as sensor or label noise.…”

Simultaneous exercise recognition and evaluation in prescribed routines: Approach to virtual coaches

“…Most of studies that fuse data from various sensors combine gyroscopes and accelerometers [8], [33]- [46], [46]- [115] or both sensors with magnetometers [116]- [152]. Few studies join the accelerometer and magnetometer data [153], [154] and only one uses the gyroscope and magnetometer data [155].…”

“…These DNNs can use just the information of accelerometers and the orientation of a set of body segments to estimate the whole-body posture [29] or fuse the information of specific force with the turn rate to estimate the joint angles [53], [69], [86], [103]. A less common approach includes the fusion of gyroscopes and magnetometers to estimate the joint angles [155]. LSTMs can also be used to estimate the orientation of the whole-body joints using the orientation obtained with sparse commercial sensors [148].…”

Self-Calibrating Magnetometer-Free Inertial Motion Tracking of 2-DoF Joints

“…The third approach is to ask the subject to assume precisely defined postures or perform a sequence of precisely defined motions. In the simplest form, this consists of a single pose calibration, often in the N-pose or T-pose [19][20][21][22], and requires magnetometers in order to be able to define two axes from one pose. A magnetometer-free alternative is to use two poses, e.g., one standing up and one lying down [23], or to derive the anatomical axes from angular rate measurements of precisely defined motions, typically around the functional axes of the joint [24][25][26].…”