Estimating Full Regional Skeletal Muscle Fibre Orientation from B-Mode Ultrasound Images Using Convolutional, Residual, and Deconvolutional Neural Networks

Cunningham, Ryan; Sánchez, María Beatriz González; May, Gregory D.; Loram, Ian D.

doi:10.3390/jimaging4020029

Cited by 40 publications

(36 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Application to US is under-developed and application to skeletal muscle is rare. This study, builds upon previous work by our group realizing the scientific and clinical value of in-vivo skeletal muscle analysis [13]- [18], applying deep learning to skeletal muscle US [19]- [21] and This diagram shows a residual encode-decoder network with a spatial SoftMax classification layer. On the far left, the raw ultrasound image is input to the encoder network.…”

Section: B Contribution Of This Studymentioning

confidence: 99%

“…The application of machine learning and specifically deep learning to analysis of ultrasound images of muscle is rare [19], [21], [26]. While under developed, the domain of muscle diagnosis is valuable since unlike visual observation, manual Using principal components of US segment boundaries from all participants (35 dystonia, 26 age matched controls), we have assessed the extent to which shape alone can distinguish participants with dystonia from those without dystonia.…”

Section: Relationship To Previous Workmentioning

confidence: 99%

“…Third, this work predicts and interprets muscle shape, excluding prediction of texture and muscle activity. Further work will exploit the ultrasound information content revealing muscle function [19], [20] as well as geometry.…”

Section: E Limitationsmentioning

confidence: 99%

See 2 more Smart Citations

Automated analysis of neck muscle boundaries for cervical dystonia using ultrasound imaging and deep learning

Loram¹,

Siddique²,

Sánchez³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Objective: To provide an automated visualization, pattern analysis and classification of neck muscle boundaries comparing cervical dystonia with healthy controls. Methods: We recorded transverse cervical ultrasound (US) images and whole-body motion analysis of sixty-one standing participants (35 cervical dystonia, 26 age matched controls). We manually annotated 3,100 images sampling a range of pitch, yaw, and roll head movements (2,000 this dataset, 1,100 previous dataset of 28 healthy participants), and trained, validated and tested a deep residual deconvolutional neural network (DCNN) to classify pixels to 13 categories (five paired neck muscles, skin, ligamentum nuchae, vertebra). For all participants in their normal standing posture, we generated held out DCNN segmented US images, extracted segmentation boundaries, transformed the boundaries to principal components and clustered the dystonia participants into two groups using their component scores. Results: For all segments, metrics of segmentation accuracy were Jaccard Index (50±21%) and Hausdorff Distance (5.8±4 mm). For the four deep muscle layers, their boundaries were used to predict central injection sites for each muscle with average precision 94±2%. Using 10-fold cross-validation to select predictive components, linear analysis of component scores identified two significant eigen-patterns discriminating Dystonia from Controls and classifying group membership (Dystonia1, Dystonia2, Control) correctly at 93.4%. Motion analysis showed the groups differed significantly according to head yaw-rotation posture and truncal posture. Conclusion: Neck muscle shape alone discriminates dystonia from healthy controls. Significance: Using deep learning, US imaging allows online, automated visualization, pattern analysis, diagnostic classification of cervical dystonia and segmentation of individual muscles for targeted injection.

show abstract

Section: B Contribution Of This Studymentioning

confidence: 99%

Section: Relationship To Previous Workmentioning

confidence: 99%

See 1 more Smart Citation

Automated analysis of neck muscle boundaries for cervical dystonia using ultrasound imaging and deep learning

Loram¹,

Siddique²,

Sánchez³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Image segmentation can be very challenging [6]- [14], particularly medical image segmentation [15], [16], [25]- [28], [17]- [24]. Recently complicated segmentation tasks [29]- [34], in medical imaging [35]- [42], and US [43]- [46], but not in skeletal muscle US, though there are some applications to muscle US [47]- [49]. The lack of publicly available labelled data, benchmark methods and results hinder the development of this domain.…”

Section: Introductionmentioning

confidence: 99%

Ultrasound Segmentation of Cervical Muscle during head motion: a Dataset and a Benchmark using Deconvolutional Neural Networks

Cunningham¹,

Sánchez²,

Loram³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Objectives: To automate online segmentation of cervical muscles from transverse ultrasound (US) images of the human neck during functional head movement. To extend ground-truth labelling methodology beyond dependence upon MRI imaging of static head positions required for application to participants with involuntary movement disorders. Method: We collected sustained sequences (> 3 minutes) of US images of human posterior cervical neck muscles at 25 fps from 28 healthy adults, performing visually-guided pitch and yaw head motions. We sampled 1,100 frames (approx. 40 per participant) spanning the experimental range of head motion. We manually labelled all 1,100 US images and trained deconvolutional neural networks (DCNN) with a spatial SoftMax regression layer to classify every pixel in the full resolution (525x491) US images, as one of 14 classes (10 muscles, ligamentum nuchae, vertebra, skin, background). We investigated ‘MaxOut’ and Exponential Linear unit (ELU) transfer functions and compared with our previous benchmark (analytical shape modelling). Results: These DCNNs showed higher Jaccard Index (53.2%) and lower Hausdorff Distance (5.7 mm) than the previous benchmark (40.5%, 6.2 mm). SoftMax Confidence corresponded with correct classification. ‘MaxOut’ outperformed ELU marginally. Conclusion: The DCNN architecture accommodates challenging images and imperfect manual labels. The SoftMax layer gives user feedback of likely correct classification. The ‘MaxOut’ transfer function benefits from near-linear operation, compatibility with deconvolution operations and the dropout regulariser. Significance: This methodology for labelling ground-truth and training automated labelling networks is applicable for dynamic segmentation of moving muscles and for participants with involuntary movement disorders who cannot remain still.

show abstract

“…To retain the precise detection and tracking of point-features, [14], [18], [24], [25] one solution is to utilize deconvolutional layers and max-un-pooling layers [26], [27]. These layers, through learning, reverse the process of convolution, and recover the position information that was lost during max-pooling, to reconstruct a precise pathway to the full resolution image [13], [16], [17], [28]- [32]. However, the training time and runtime is relatively expensive.…”

mentioning

confidence: 99%

Fully automated image-based estimation of postural point-features in children with cerebral palsy using deep learning

Cunningham¹,

Sánchez²,

Butler³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

Objective: To automate identification of postural point-features from colour videos of children with neuromotor disability, during clinical assessment. The automatic identification of 13 points of interest (2, 6, 2, 3 points on the head, trunk, pelvis, arm respectively) is required to estimate the location and orientation of head, trunk, and arm segments, from videos of the clinical test "Segmental Assessment of Trunk Control" (SATCo) which is a test of seated postural control. Methods: Three expert operators manually annotated 13 point-features in every fourth image of 177 short (5-10 second) videos (25 Hz) showing 12 children with cerebral palsy (ages: 4.52 ±2.4, male: 9), participating in SATCo testing. Linear interpolation for the remaining images resulted in 30,825 annotated images. Meanpooling and max-pooling convolutional neural networks were trained with cross-validation, giving held-out test results for all children. Results: The point-features were estimated with error 4.40 ±3.75 pixels (mean-pooling), and 4.49 ±4.45 pixels (maxpooling), at approximately 100 images per second. Trunk segment angles (head, neck, 6 thoraco-lumbar-pelvic segments) were estimated with error 6.4° ±2.8° allowing accurate classification (F1 > 80%) of deviation from a reference posture at thresholds up to 3°, 3°, 2° respectively. Contact between arm point features (elbow, wrist) and supporting surface was classified at F1 = 80.5%. Conclusion and Significance: This study demonstrates, for the first time, a technical solution to automate identification of i) a sitting segmental-posture including individual trunk segments, ii) changes away from that posture, and iii) support from the upper limb, required for the clinical SATCo.

show abstract

Estimating Full Regional Skeletal Muscle Fibre Orientation from B-Mode Ultrasound Images Using Convolutional, Residual, and Deconvolutional Neural Networks

Cited by 40 publications

References 30 publications

Automated analysis of neck muscle boundaries for cervical dystonia using ultrasound imaging and deep learning

Automated analysis of neck muscle boundaries for cervical dystonia using ultrasound imaging and deep learning

Ultrasound Segmentation of Cervical Muscle during head motion: a Dataset and a Benchmark using Deconvolutional Neural Networks

Fully automated image-based estimation of postural point-features in children with cerebral palsy using deep learning

Contact Info

Product

Resources

About