Training Data Selection and Optimal Sensor Placement for Deep-Learning-Based Sparse Inertial Sensor Human Posture Reconstruction

Abstract

Although commercial motion-capture systems have been widely used in various applications, the complex setup limits their application scenarios for ordinary consumers. To overcome the drawbacks of wearability, human posture reconstruction based on a few wearable sensors have been actively studied in recent years. In this paper, we propose a deep-learning-based sparse inertial sensor human posture reconstruction method. This method uses bidirectional recurrent neural network (Bi-RNN) to build an a priori model from a large motion dataset to build human motion, thereby the low-dimensional motion measurements are mapped to whole-body posture. To improve the motion reconstruction performance for specific application scenarios, two fundamental problems in the model construction are investigated: training data selection and sparse sensor placement. The problem of deep-learning training data selection is to select independent and identically distributed (IID) data for a certain scenario from the accumulated imbalanced motion dataset with sufficient information. We formulate the data selection into an optimization problem to obtain continuous and IID data segments, which comply with a small reference dataset collected from the target scenario. A two-step heuristic algorithm is proposed to solve the data selection problem. On the other hand, the optimal sensor placement problem is studied to exploit most information from partial observation of human movement. A method for evaluating the motion information amount of any group of wearable inertial sensors based on mutual information is proposed, and a greedy searching method is adopted to obtain the approximate optimal sensor placement of a given sensor number, so that the maximum motion information and minimum redundancy is achieved. Finally, the human posture reconstruction performance is evaluated with different training data and sensor placement selection methods, and experimental results show that the proposed method takes advantages in both posture reconstruction accuracy and model training time. In the 6 sensors configuration, the posture reconstruction errors of our model for walking, running, and playing basketball are 7.25, 8.84, and 14.13, respectively.

Simultaneous exercise recognition and evaluation in prescribed routines: Approach to virtual coaches

“…They have replaced the RNNs which suffer from the vanishing gradient problem during the training and include forget gates to quickly adapt to the new changes of data. These DNNs can use just the information of accelerometers and the orientation of a set of body segments to estimate the whole-body posture [29] or fuse the information of specific force with the turn rate to estimate the joint angles [53], [69], [86], [103]. A less common approach includes the fusion of gyroscopes and magnetometers to estimate the joint angles [155].…”

“…the joint or segment orientation or location. In this review, we found 26 works that use reference data, that can be obtained from a stereophotogrammetric system (17/26) [21], [27], [34], [49], [53], [54], [56], [58], [68], [69], [86], [90], [91], [103], [111], [114], [115], electro-goniometer and encoders (2/26) [71], [84] or inertial sensors (7/26) [29], [100], [109], [134], [147], [148], [155]. Fig.…”

“…Sensor fusion algorithms use one IMU per segment to monitor the whole-body posture or model biomechanical relationships between segments to reduce this number with different constraints. Conversely, the approach in ML-based proposals focused on the whole-body posture is the optimization of the number of devices with the use of the socalled sparse IMUs, as in [29], [58], [147], [148], [155]. This approach is also used to monitor specific limbs, as legs, reducing the number of sensors [21], [100], [134].…”

3D Knee and Hip Angle Estimation With Reduced Wearable IMUs via Transfer Learning During Yoga, Golf, Swimming, Badminton, and Dance

“…W EARABLE lower-limb joint angle estimation using a reduced inertial measurement unit (IMU) sensor set facilitates the rapid and cost-effective assessment of sports injury risk [1]- [5], motion capture [6]- [9], and improvement of motion techniques [1], [10], [11]. The IMU is a commonly used, lightweight, and affordable sensor that is often integrated with machine learning models to enable kinematic estimation [6], [10].…”

Optimal Sensor Placement Using Learning Models—A Mediterranean Case Study

“…Recently, the quality of reconstruction also emerged as a new criterion for feature selection [4][5][6]. Regardless of whether we observe this problem in terms of sensor (site) selection or optimal sensor placement, we can find a number of proposed solution [7][8][9][10][11][12][13]. This is even more true when we consider the different feature selection algorithms available (e.g., [14][15][16][17]).…”