A Comparison of Three Neural Network Approaches for Estimating Joint Angles and Moments from Inertial Measurement Units

Gait analysis based on inertial sensors has become an effective method of quantifying movement mechanics, such as joint kinematics and kinetics. Machine learning techniques are used to reliably predict joint mechanics directly from streams of IMU signals for various activities. These data-driven models require comprehensive and representative training datasets to be generalizable across the movement variability seen in the population at large. Bottlenecks in model development frequently occur due to the lack of sufficient training data and the significant time and resources necessary to acquire these datasets. Reliable methods to generate synthetic biomechanical training data could streamline model development and potentially improve model performance. In this study, we developed a methodology to generate synthetic kinematics and the associated predicted IMU signals using open source musculoskeletal modeling software. These synthetic data were used to train neural networks to predict three degree-of-freedom joint rotations at the hip and knee during gait either in lieu of or along with previously measured experimental gait data. The accuracy of the models’ kinematic predictions was assessed using experimentally measured IMU signals and gait kinematics. Models trained using the synthetic data out-performed models using only the experimental data in five of the six rotational degrees of freedom at the hip and knee. On average, root mean square errors in joint angle predictions were improved by 38% at the hip (synthetic data RMSE: 2.3°, measured data RMSE: 4.5°) and 11% at the knee (synthetic data RMSE: 2.9°, measured data RMSE: 3.3°), when models trained solely on synthetic data were compared to measured data. When models were trained on both measured and synthetic data, root mean square errors were reduced by 54% at the hip (measured + synthetic data RMSE: 1.9°) and 45% at the knee (measured + synthetic data RMSE: 1.7°), compared to measured data alone. These findings enable future model development for different activities of clinical significance without the burden of generating large quantities of gait lab data for model training, streamlining model development, and ultimately improving model performance.

Section: Discussionmentioning

confidence: 99%

The Use of Synthetic IMU Signals in the Training of Deep Learning Models Significantly Improves the Accuracy of Joint Kinematic Predictions

Renani

Eustace

Myers

et al. 2021

“…The NN had the same structure as the NN used in the LSTM model. The CNN used the structure used in the precedent studies [ 20 , 21 , 22 , 23 ]. The convolution layer used 8 sets of 15 × 1 × 1 three-dimensional matrices as a kernel, and the stride was composed of a 1 × 1 × 1 matrix.…”

Section: Methods and Design Of The Proposed Systemmentioning

confidence: 99%

“…The acceleration sensor, the angular velocity sensor, the height sensor, and the EMG sensor are used to measure the current movement, running, walking, and climbing stairs [ 5 , 6 , 7 , 8 , 9 , 10 ]. Based on these studies, research on motion measurement using artificial intelligence is also being conducted, which greatly contributes to the development of the healthcare and game industry [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 ]. Research related to motorcycle accidents is also increasing.…”

Section: Introductionmentioning

confidence: 99%

A Study on the Application of LSTM to Judge Bike Accidents for Inflating Wearable Airbags

Woo

Byun

et al. 2021

The traffic accident occurrence rate is increasing relative to the increase in the number of people using personal mobility device (PM). This paper proposes an airbag system with a more efficient algorithm to decide the deployment of a wearable bike airbag in case of an accident. The existing wearable airbags are operated by judging the accident situations using the thresholds of sensors. However, in this case, the judgment accuracy can drop against various motions. This study used the long short-term memory (LSTM) model using the sensor values of the inertial measurement unit (IMU) as input values to judge accident occurrences, which obtains data in real time from the three acceleration-axis and three angular velocity-axis sensors on the driver motion states and judges whether or not an accident has occurred using the obtained data. The existing neural network (NN) or convolutional neural network (CNN) model judges only the input data. This study confirmed that this model has a higher judgment accuracy than the existing NN or CNN by giving strong points even in “past information” through LSTM by regarding the driver motion as time-series data.

“…The LSTM cell has a more complex structure and was shown to better handle long-term dependencies within the data [33]. LSTM networks have been proposed to solve problems dealing with wearable inertial sensor data, e.g., in the area of odometry [24,25], pedestrian dead reckoning [26], kinematics [15,31,34,35], attitude estimation [32,36], fall risk assessment [30], activity recognition [37,38], among others.…”

Section: Network Architecturementioning

confidence: 99%

“…The network architecture used in this study was inspired by past works in related fields, namely, in [25], [34] or [35]. In kinematics, in particular, LSTM networks were proposed to predict multidimensional sequences of kinematic variables-e.g., joint angles and moments-from multidimensional sequences of inertial sensor data-acceleration and angular rotation [34,35]. Using a similar approach, we proposed to predict multidimensional trajectories from multidimensional IMU data.…”

Section: Problem Formulationmentioning

confidence: 99%

A Deep Learning Approach for Foot Trajectory Estimation in Gait Analysis Using Inertial Sensors

Guimar�ães

Sousa

Correia

2021

Gait performance is an important marker of motor and cognitive decline in older adults. An instrumented gait analysis resorting to inertial sensors allows the complete evaluation of spatiotemporal gait parameters, offering an alternative to laboratory-based assessments. To estimate gait parameters, foot trajectories are typically obtained by integrating acceleration two times. However, to deal with cumulative integration errors, additional error handling strategies are required. In this study, we propose an alternative approach based on a deep recurrent neural network to estimate heel and toe trajectories. We propose a coordinate frame transformation for stride trajectories that eliminates the dependency from previous strides and external inputs. Predicted trajectories are used to estimate an extensive set of spatiotemporal gait parameters. We evaluate the results in a dataset comprising foot-worn inertial sensor data acquired from a group of young adults, using an optical motion capture system as a reference. Heel and toe trajectories are predicted with low errors, in line with reference trajectories. A good agreement is also achieved between the reference and estimated gait parameters, in particular when turning strides are excluded from the analysis. The performance of the method is shown to be robust to imperfect sensor-foot alignment conditions.