Motion Inference Using Sparse Inertial Sensors, Self-Supervised Learning, and a New Dataset of Unscripted Human Motion

Abstract

In recent years, wearable sensors have become common, with possible applications in biomechanical monitoring, sports and fitness training, rehabilitation, assistive devices, or human-computer interaction. Our goal was to achieve accurate kinematics estimates using a small number of sensors. To accomplish this, we introduced a new dataset (the Virginia Tech Natural Motion Dataset) of full-body human motion capture using XSens MVN Link that contains more than 40 h of unscripted daily life motion in the open world. Using this dataset, we conducted self-supervised machine learning to do kinematics inference: we predicted the complete kinematics of the upper body or full body using a reduced set of sensors (3 or 4 for the upper body, 5 or 6 for the full body). We used several sequence-to-sequence (Seq2Seq) and Transformer models for motion inference. We compared the results using four different machine learning models and four different configurations of sensor placements. Our models produced mean angular errors of 10–15 degrees for both the upper body and full body, as well as worst-case errors of less than 30 degrees. The dataset and our machine learning code are freely available.

Dimensionality Reduction of Human Gait for Prosthetic Control

“…Indeed, this discussion focuses on facilitation of “high-level” prosthetic control, or mode selection, rather than “low-level” control over moment to moment commands to the actuator which remains critically important. Furthermore, our results may still be replicable using raw inertial measurement unit data from a few strategically placed sensors on the lower limbs, rather than the full wearable motion capture system used here, in line with the work by Geissinger and Asbeck (2020b) .…”

“…We are also using the Virginia Tech Natural Motion Dataset Geissinger and Asbeck (2020a) , also collected using an Xsens system. It contains 40 h of natural, unscripted movement from 17 participants, including 13 participants on a college campus and four participants working in a home improvement store.…”

“…Pettee et al (2019) used such techniques to produce manifolds of human dancing, from which samples of novel dance choreography may be decoded. Geissinger and Asbeck (2020b) utilized similar principles to infer complete joint information from sparse sensor input on the natural motion dataset considered here.…”

Radar Recorded Child Vital Sign Public Dataset and Deep Learning-Based Age Group Classification Framework for Vehicular Application

“…Inspired by the visual cortex of animals, the CNNs works on the grid data (2D/3D images). A typical CNN networks is a layered network consisting of three main layers: 1) input layer, the hidden layer ant the classification layer [ 14 ]. A hidden layer in CNN further comprise of a convolution layer, the max-pooling layer, and batch normalization layer.…”

“…In addition to that, Riquelme et al [ 13 ] presented an infrared sensor dataset of fall detection. Similarly Geissinger and Asbeck [ 14 ] in 2020 recently proposed a human motion dataset using inertial sensors. Another similar activity recognition dataset was proposed in [ 15 ].…”