Estimation of absolute states of human skeletal muscle via standard B-mode ultrasound imaging and deep convolutional neural networks

Abstract

Objective: To test automated in vivo estimation of active and passive skeletal muscle states using ultrasonic imaging. Background: Current technology (electromyography, dynamometry, shear wave imaging) provides no general, non-invasive method for online estimation of skeletal intramuscular states. Ultrasound (US) allows non-invasive imaging of muscle, yet current computational approaches have never achieved simultaneous extraction nor generalisation of independently varying, active and passive states. We use deep learning to investigate the generalizable content of 2D US muscle images. Method: US data synchronized with electromyography of the calf muscles, with measures of joint moment/angle were recorded from 32 healthy participants (7 female, ages: 27.5, 19-65). We extracted a region of interest of medial gastrocnemius and soleus using our prior developed accurate segmentation algorithm. From the segmented images, a deep convolutional neural network was trained to predict three absolute, drift-free, components of the neurobiomechanical state (activity, joint angle, joint moment) during experimentally designed, simultaneous, independent variation of passive (joint angle) and active (electromyography) inputs. Results: For all 32 held-out participants (16-fold cross-validation) the ankle joint angle, electromyography, and joint moment were estimated to accuracy 55±8%, 57±11%, and 46±9% respectively. Significance: With 2D US imaging, deep neural networks can encode in generalizable form, the activity-length-tension state relationship of muscle. Observation only, low power, 2D US imaging can provide a new category of technology for non-invasive estimation of neural output, length and tension in skeletal muscle. This proof of principle has value for personalised muscle diagnosis in pain, injury, neurological conditions, neuropathies, myopathies and ageing.

A deep learning method to predict ankle joint moment during walking at different speeds with ultrasound imaging: A framework for assistive devices control

“…well as to estimate/predict states generally from individual muscles, given the fact that the information is well encoded in skeletal muscles' collagen structure and are observable by using US imaging (Cunningham and Loram, 2020). Recent research studies have also applied US imaging þ deep (machine) learning to estimate skeletal muscles' activation levels (Cunningham et al, 2017b;Cunningham and Loram, 2020;Feigin et al, 2020), fascicle length (Rosa et al, 2021), fascicle orientation (Cunningham et al, 2017a), and muscle segmentation (Carneiro and Nascimento, 2013;Zhou et al, 2020).…”

“…Therefore, the motivation of this work is to investigate the continuous joint volitional effort prediction for lower limb joint functionalities by using high-dimensional features from US imaging and a deep learning method. To the best of our knowledge, only a few recent studies have investigated deep learning approaches for continuous ankle joint kinematics, kinetics, and muscle state estimation (Cunningham et al, 2017b;Cunningham and Loram, 2020). However, they only focused on the active and passive ankle joint movement tasks at the standing posture, and no functional dynamic locomotion tasks were discussed.…”

“…In addition, as mentioned by Saxby et al (2020), the processing of US imaging data to derive the aforementioned explicit features is relatively subjective, time-consuming, and tedious. Therefore, more advanced deep learning approaches have been developed to deal with the high-dimensional explicit and/or implicit features from either sEMG or US signals for continuous joint kinematics and/or kinetics estimation (indicating human motion intent or volitional effort) (Cunningham et al, 2017b;Liu et al, 2019;Cunningham and Loram, 2020;Ma et al, 2020;Xu et al, 2020;Yu et al, 2020;Pancholi et al, 2021). For example, in Xu et al (2020), parallel convolutional neural networks (CNNs) were used to continuously predict an individual finger's force in real-time based on the temporal energy heatmap and frequency spectrum map from high-density sEMG signals, which outperformed the prediction performance by using a motor unit decomposition method and a conventional sEMG amplitude-based method.…”

Machine learning methods to support personalized neuromusculoskeletal modelling

“…Another recent study into the utility of neural networks has been to use simple twodimensional ultrasound to estimate muscle states during passive and active contractions (Cunningham and Loram 2020). The authors trained a convolutional neural network based on inputs of joint angle, moment, and EMG with the associated ultrasound image of the muscle.…”

Multimodal assessment of spasticity using a point-of-care instrumented glove to separate neural and biomechanical contributions

“…EMG is meant to capture involuntary muscle activations, but this technique suffers from low signal reproducibility due to issues with electrode placement and motion artifacts ( Misgeld et al., 2016 ; Sloot et al., 2017 ; Yu et al., 2020 ; Wang et al., 2017 ). Ultrasound has been used to inspect muscle fiber lengths and cross-sectional area that relate to muscle strength ( Moreau et al., 2009 ; Cunningham and Loram, 2020 ). Yet this method does not address the motion-dependent aspects of spasticity.…”