DeepBBWAE-Net: A CNN-RNN Based Deep SuperLearner for Estimating Lower Extremity Sagittal Plane Joint Kinematics Using Shoe-Mounted IMU Sensors in Daily Living

Hossain, Md Sanzid Bin; Dranetz, Joseph; Choi, Hwan; Guo, Zhishan

doi:10.1109/jbhi.2022.3165383

Cited by 23 publications

(29 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, to ensure practicality and users' comfort, we need to minimize the number of IMU sensors. More specifically, if we can implement only shoe-mounted sensors similar to kinematics estimation in [54], it would be more helpful to maintain sensors. Since single limb joint moments and GRFs are affected by contralateral limb during early stance phase and terminal stance phase, incorporating IMU information on both limbs would gather more meaningful knowledge of walking dynamics, and this can improve the prediction further.…”

Section: Discussionmentioning

confidence: 99%

Estimation of Lower Extremity Joint Moments and 3D Ground Reaction Forces Using IMU Sensors in Multiple Walking Conditions: A Deep Learning Approach

Hossain

Guo

Choi

2023

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

Human kinetics, specifically joint moments and ground reaction forces (GRFs) can provide important clinical information and can be used to control assistive devices. Traditionally, collection of kinetics is mostly limited to the lab environment because it relies on data that measured from a motion capture system and floor-embedded force plates to calculate the dynamics via musculoskeletal models. This spatially limited method makes it extremely challenging to measure kinetics outside the laboratory in a variety of walking conditions due to the expensive device setup and large space required. Recently, employing machine learning with IMU sensors are suggested as an alternative method for biomechanical analyses. Although these methods enable estimating human kinetic data outside the laboratory by linking IMU sensor data with kinetics dataset, they were limited to show inaccurate kinetic estimates even in highly repeatable single walking conditions due to the employment of generic deep learning algorithms. Thus, this paper proposes a novel deep learning model, Kinetics-FM-DLR-Ensemble-Net to estimate the hip, knee, and ankle joint moments in the sagittal plane and 3 dimensional ground reaction forces (GRFs) using three IMU sensors on the thigh, shank, and foot under several representative walking conditions in daily living, such as treadmill, level-ground, stair, and ramp with different walking speeds. This is the first study that implements both joint moments and GRFs in multiple walking conditions using IMU sensors via deep learning. Our deep learning model is versatile and accurate for identifying human kinetics across diverse subjects and walking conditions and our model outperforms state-of-the-art deep learning model for kinetics estimation by a large margin.

show abstract

Section: Discussionmentioning

confidence: 99%

Estimation of Lower Extremity Joint Moments and 3D Ground Reaction Forces Using IMU Sensors in Multiple Walking Conditions: A Deep Learning Approach

Hossain

Guo

Choi

2023

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

show abstract

“…In [69], LSTMs are combined with CNN to estimate the joint angles. CNNs are also used to obtain the joint angles only using the accelerometer data [21] or fusing the gyroscope and accelerometer data [56], [111], [114]. In [34], Mundt et al made a comparison of these previous methods, CNNs and LSTMs, together with MLPs for the estimation of joint orientation.…”

Section: Adopted Algorithmsmentioning

confidence: 99%

“…the joint or segment orientation or location. In this review, we found 26 works that use reference data, that can be obtained from a stereophotogrammetric system (17/26) [21], [27], [34], [49], [53], [54], [56], [58], [68], [69], [86], [90], [91], [103], [111], [114], [115], electro-goniometer and encoders (2/26) [71], [84] or inertial sensors (7/26) [29], [100], [109], [134], [147], [148], [155]. Fig.…”

Section: Adopted Algorithmsmentioning

confidence: 99%

Simultaneous exercise recognition and evaluation in prescribed routines: Approach to virtual coaches

Villa

Casillas-Pérez

Martín

et al. 2022

Expert Systems with Applications

View full text Add to dashboard Cite

“…Therefore, our aim is to create an IMU-based motion analysis approach that is as unobtrusive as possible by reducing the number of necessary sensors. Such systems have been developed to provide spatiotemporal (e.g., [1]), sagittal plane kinematic [24][25][26], three dimensional kinematic [27] or sagittal plane kinetic [28] gait variables. These methods do not provide a comprehensive analysis considering spatiotemporal, kinematic, and kinetics characteristics of gait.…”

Section: Introductionmentioning

confidence: 99%

“…These methods do not provide a comprehensive analysis considering spatiotemporal, kinematic, and kinetics characteristics of gait. On the one hand, neural networks have been investigated to estimate specific quantities of interest using only two IMUs, mounted on the left and right shank [24] or the feet [1, 26] without considering physical correctness. On the other hand, physics-based optimization has been proposed to track the orientation of shank-mounted sensors with a torque-driven model [28] or to track the signals of six IMUs with a human body shape model [27].…”

Section: Introductionmentioning

confidence: 99%

Comparing sparse inertial sensor setups for sagittal-plane walking and running reconstructions

Dorschky

Nitschke

Mayer

et al. 2023

Preprint

View full text Add to dashboard Cite

A sensor setup with a small number of sensors, i.e., a sparse sensor setup, could enable a comprehensive unobtrusive motion analysis using inertial motion capture (IMC) with little obtrusion to the user, which is critical for clinical and sports applications, such as assessing disease, providing biofeedback in rehabilitation, or enhancing performance in sports. When sparse sensor setups were used, the motion analysis was either not comprehensive or physical correctness was not considered. Furthermore, the relationship between sensor setup and the resulting accuracy of spatiotemporal, kinematic, or kinetic variables has not been investigated systematically. Therefore, we investigated the accuracy of biomechanical gait reconstructions from different sparse inertial sensor setups, in which the number of sensors is reduced by not placing sensors on all body segments of interest. We created biomechanical reconstructions by solving optimal control problems with a sagittal plane musculoskeletal model. The resulting simulations tracked raw accelerometer and gyroscope data while minimizing muscular effort. We applied this approach to six different sparse sensor setups to investigate the reconstruction quality of spatiotemporal, kinematic, and kinetic variables for each of these setups at three walking and three running speeds. We found that correlations between IMC and optical motion capture (OMC) were large for all sensor setups, but that those setups without a pelvis sensor led to a forward trunk lean that was too high, as well as unrealistic kinetics and kinematics. Spatiotemporal variable estimations generally also benefitted from a pelvis sensor, but not as much as the kinetic and kinematic estimations. We found that for setups including a pelvis sensor, there was no clear benefit for including additional sensors. While walking speed, step length, and joint kinematics generally benefited from using more sensors, the hip moment and knee moment did not. During running, the ankle moment and vertical ground reaction force were even estimated more accurately with smaller sensor setups. Therefore, we conclude that sagittal-plane reconstruction of walking and running kinetics and kinematics using sparse sensor setups can improve usability at the cost of only minor reductions in accuracy. When using a sparse sensor setup for gait reconstructions using optimal control, we advise to always include a pelvis sensor to achieve accurate results.

show abstract

DeepBBWAE-Net: A CNN-RNN Based Deep SuperLearner for Estimating Lower Extremity Sagittal Plane Joint Kinematics Using Shoe-Mounted IMU Sensors in Daily Living

Cited by 23 publications

References 41 publications

Estimation of Lower Extremity Joint Moments and 3D Ground Reaction Forces Using IMU Sensors in Multiple Walking Conditions: A Deep Learning Approach

Estimation of Lower Extremity Joint Moments and 3D Ground Reaction Forces Using IMU Sensors in Multiple Walking Conditions: A Deep Learning Approach

Simultaneous exercise recognition and evaluation in prescribed routines: Approach to virtual coaches

Comparing sparse inertial sensor setups for sagittal-plane walking and running reconstructions

Contact Info

Product

Resources

About