The Application of Deep Convolutional Neural Networks to Ultrasound for Modelling of Dynamic States within Human Skeletal Muscle

Abstract

This paper concerns the fully automatic direct in vivo measurement of active and passive dynamic skeletal muscle states using ultrasound imaging. Despite the long standing medical need (myopathies, neuropathies, pain, injury, ageing), currently technology (electromyography, dynamometry, shear wave imaging) provides no general, non-invasive method for online estimation of skeletal intramuscular states. Ultrasound provides a technology in which static and dynamic muscle states can be observed non-invasively, yet current computational image understanding approaches are inadequate. We propose a new approach in which deep learning methods are used for understanding the content of ultrasound images of muscle in terms of its measured state. Ultrasound data synchronized with electromyography of the calf muscles, with measures of joint torque/angle were recorded from 19 healthy participants (6 female, ages: 30 ± 7.7). A segmentation algorithm previously developed by our group was applied to extract a region of interest of the medial gastrocnemius. Then a deep convolutional neural network was trained to predict the measured states (joint angle/torque, electromyography) directly from the segmented images. Results revealed for the first time that active and passive muscle states can be measured directly from standard b-mode ultrasound images, accurately predicting for a held out test participant changes in the joint angle, electromyography, and torque with as little error as 0.022°, 0.0001V, 0.256Nm (root mean square error) respectively.

Ultrasound imaging links soleus muscle neuromechanics and energetics during human walking with elastic ankle exoskeletons

“…This remains a formidable challenge in humans where direct measurements of underlying muscle dynamics are so far, too invasive. However, there is potential to use advanced tools from image processing and machine-learning to extract real-time muscle length and velocity samples from ultrasound images 82 , and then use them in combination with EMG or a musculoskeletal model 83,84 as input to an exoskeleton controller. Access to muscle dynamics in real-time would enable application of state-of-the-art human in the loop optimization techniques 17,85,86 to individual or groups of target muscles.…”

The Feasibility of Dynamic Musculoskeletal Function Analysis of the Vastus Lateralis in Endurance Runners Using Continuous, Hands-Free Ultrasound

“…Additionally, the currently used algorithm could be improved further by taking into account an estimation of the aponeurosis motion during a gait cycle based on the successfully analyzed frames of (other) gait cycles. Alternatively, deep learning models could be used for feature detection [8,37].…”

Ultrasound Segmentation of Cervical Muscle during head motion: a Dataset and a Benchmark using Deconvolutional Neural Networks

“…Participants were instructed to allow their head to turn within a comfortable range to follow the target with the tip of their nose. "Horizontal", "vertical" and "combined" trials contained respectively horizontal motion of the target, vertical motion of the target and a combination of independent horizontal and vertical components similar to [49]. These independent components included sines of multiple frequencies leading to transiently correlated and uncorrelated intervals.…”

“…Image segmentation can be very challenging [6]- [14], particularly medical image segmentation [15], [16], [25]- [28], [17]- [24]. Recently complicated segmentation tasks [29]- [34], in medical imaging [35]- [42], and US [43]- [46], but not in skeletal muscle US, though there are some applications to muscle US [47]- [49]. The lack of publicly available labelled data, benchmark methods and results hinder the development of this domain.…”