A deep learning method to predict ankle joint moment during walking at different speeds with ultrasound imaging: A framework for assistive devices control

Abstract

Robotic assistive or rehabilitative devices are promising aids for people with neurological disorders as they help regain normative functions for both upper and lower limbs. However, it remains challenging to accurately estimate human intent or residual efforts non-invasively when using these robotic devices. In this article, we propose a deep learning approach that uses a brightness mode, that is, B-mode, of ultrasound (US) imaging from skeletal muscles to predict the ankle joint net plantarflexion moment while walking. The designed structure of customized deep convolutional neural networks (CNNs) guarantees the convergence and robustness of the deep learning approach. We investigated the influence of the US imaging’s region of interest (ROI) on the net plantarflexion moment prediction performance. We also compared the CNN-based moment prediction performance utilizing B-mode US and sEMG spectrum imaging with the same ROI size. Experimental results from eight young participants walking on a treadmill at multiple speeds verified an improved accuracy by using the proposed US imaging + deep learning approach for net joint moment prediction. With the same CNN structure, compared to the prediction performance by using sEMG spectrum imaging, US imaging significantly reduced the normalized prediction root mean square error by 37.55% ( $ p $ < .001) and increased the prediction coefficient of determination by 20.13% ( $ p $ < .001). The findings show that the US imaging + deep learning approach personalizes the assessment of human joint voluntary effort, which can be incorporated with assistive or rehabilitative devices to improve clinical performance based on the assist-as-needed control strategy.

A CNN-LSTM model for six human ankle movements classification on different loads

“…Stroke is one of the most common diseases among older people, and motor impairments of limbs, such as foot drop and spastic equinovarus foot deformity are one of the most common outcomes after stroke (Leardini et al, 2019 ). The ankle joint has many important functions including assisting people walking, supporting the body’s weight, maintaining balance, and changing posture (from sitting to standing or from lying to sitting; Jung, 2016 ; Zhang et al, 2022 ). Normal functioning in people’s daily lives needs a normal ankle joint; mobility deficits on the ankle joint can lead to difficulties in walking and other activities of daily life (Dettwyler et al, 2004 ; Jamwal et al, 2014 ).…”

“…To achieve this type of robot-assisted bilateral training, decoding the movement intention of the unaffected limb is necessary to control the robot placed on the affected side. Surface electromyography (sEMG) signals are widely used to decode human movement intentions during rehabilitation training (Varol et al, 2010 ; Joshi et al, 2013 ; Liu et al, 2019 ; Gautam et al, 2020 ; Zhang et al, 2021 , 2022 ) For lower-limb rehabilitation training, the current literature mainly focuses on gait recognition rather than ankle movement classification (Varol et al, 2010 ; Joshi et al, 2013 ; Liu et al, 2019 ; Gautam et al, 2020 ; Zhang et al, 2021 , 2022 ). However, gait recognition is applicable to individuals with sufficient walking ability, whereas platform-based ankle rehabilitation robots are more suitable for early stroke patients.…”

Data-efficient human walking speed intent identification

“…In contrast, pattern recognition approaches depend less on which sensors are used and more on the information content available from the data. Pattern recognition is most often pursued with long short-term memory networks (Romero-Hernandez et al, 2019 ; Kim et al, 2022 ), convolutional neural networks (Casale et al, 2011 ; Zhang et al, 2022a ), or other architectures that have proven useful for the gait intent recognition or prediction tasks (Fang et al, 2020 ). While these strategies reliably predict human joint movements, they typically require massive amounts of training data.…”