A segmentation-informed deep learning framework to register dynamic two-dimensional magnetic resonance images of the vocal tract during speech

Ruthven, Matthieu; Miquel, Marc E.; King, Andrew P.

doi:10.1016/j.bspc.2022.104290

Cited by 3 publications

(5 citation statements)

References 63 publications

(102 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…et al 2023), and the associated protocols, even when the dataset is not immediately available, serve as valuable guidelines for others to gather high-quality data (Lim et al 2023, Wu et al 2023. Notably, there is a uptick in the utilisation of machine learning techniques in tasks related to vocal tract MRI (Ribeiro et al 2022, Ruthven et al 2023. Additionally, a toolkit for assessing vocal tract shape has been developed (Belyk et al 2023).…”

Section: Nasal Airflowmentioning

confidence: 99%

Research in methodologies for modelling the oral cavity

Shahid,

French,

Valstar

et al. 2024

Biomed. Phys. Eng. Express

View full text Add to dashboard Cite

The paper aims to explore the current state of understanding surrounding in silico oral modelling. This involves exploring methodologies, technologies and approaches pertaining to the modelling of the whole oral cavity; both internally and externally visible structures that may be relevant or appropriate to oral actions. Such a model could be referred to as a “complete model” which includes consideration of a full set of facial features (i.e. not only mouth) as well as synergistic stimuli such as audio and facial thermal data. 3D modelling technologies capable of accurately and efficiently capturing a complete representation of the mouth for an individual have broad applications in the study of oral actions, due to their cost-effectiveness and time efficiency.This review delves into the field of clinical phonetics to classify oral actions pertaining to both speech and non-speech movements, identifying how the various vocal organs play a role in the articulatory and masticatory process. Vitaly, it provides a summation of 12 articulatory recording methods, forming a tool to be used by researchers in identifying which method of recording is appropriate for their work. After addressing the cost and resource-intensive limitations of existing methods, a new system of modelling is proposed that leverages external to internal correlation modelling techniques to create a more efficient models of the oral cavity.The vision is that the outcomes will be applicable to a broad spectrum of oral functions related to physiology, health and wellbeing, including speech, oral processing of foods as well as dental health. The applications may span from speech correction, designing foods for the aging population, whilst in the dental field we would be able to gain information about patient’s oral actions that would become part of creating a personalised dental treatment plan.

show abstract

Section: Nasal Airflowmentioning

confidence: 99%

Research in methodologies for modelling the oral cavity

Shahid,

French,

Valstar

et al. 2024

Biomed. Phys. Eng. Express

View full text Add to dashboard Cite

show abstract

“…In particular, there is interest in measuring the size, shape and motion of the vocal tract 18 , 19 , 22 – 32 and articulators such as the soft palate 20 , 33 – 37 . To avoid the burden of manual measurements, methods to (semi-)automatically measure the size and shape of the vocal tract have been developed 38 – 46 and methods to automatically measure the size, shape and motion of the soft palate are beginning to be developed 33 , 47 – 50 . Consistent with trends in other image analysis fields, most of the recently developed methods utilise convolutional neural networks (CNNs) and are therefore deep learning based 42 – 50 .…”

Section: Background and Summarymentioning

confidence: 99%

“…These GT segmentations are manually created, a process that is time-consuming and, particularly for biomedical images, requires input by specialists. While GT segmentations for 2D midsagittal MR images of the vocal tract have been created 46 – 50 , none are currently publicly available. The public availability of image sets with corresponding GT segmentations has been found to stimulate the development of state-of-the-art image analysis methods 54 – 56 .…”

Section: Background and Summarymentioning

confidence: 99%

“…Second, by making code and instructions to implement a current state-of-the-art deep-learning-based speech MR image analysis method 47 publicly available, this work provides the speech MRI community and others with a starting point for the development of such methods. Although, the MRI data made available has been previously used in published work; 47 , 50 neither data nor segmentations had been published. Since the previous work was published, the GT segmentations have been revised.…”

Section: Background and Summarymentioning

confidence: 99%

“…This dataset contains images of only five healthy volunteers all acquired using the same MRI scanner and sequence combination. Although the low number of subjects is a limitation, it was previously shown to be sufficient to develop and train segmentation networks 47 or to inform registration 50 . Clear instructions for acquisition and segmentations are given for readers who wish to increase the size of the dataset.…”

Section: Usage Notesmentioning

confidence: 99%

See 2 more Smart Citations

Real-time speech MRI datasets with corresponding articulator ground-truth segmentations

Ruthven,

Peplinski,

Adams

et al. 2023

Sci Data

Self Cite

View full text Add to dashboard Cite

The use of real-time magnetic resonance imaging (rt-MRI) of speech is increasing in clinical practice and speech science research. Analysis of such images often requires segmentation of articulators and the vocal tract, and the community is turning to deep-learning-based methods to perform this segmentation. While there are publicly available rt-MRI datasets of speech, these do not include ground-truth (GT) segmentations, a key requirement for the development of deep-learning-based segmentation methods. To begin to address this barrier, this work presents rt-MRI speech datasets of five healthy adult volunteers with corresponding GT segmentations and velopharyngeal closure patterns. The images were acquired using standard clinical MRI scanners, coils and sequences to facilitate acquisition of similar images in other centres. The datasets include manually created GT segmentations of six anatomical features including the tongue, soft palate and vocal tract. In addition, this work makes code and instructions to implement a current state-of-the-art deep-learning-based method to segment rt-MRI speech datasets publicly available, thus providing the community and others with a starting point for developing such methods.

show abstract

Human-inspired computational models for European Portuguese: a review

Teixeira

Silva

2023

Lang Resources & Evaluation

View full text Add to dashboard Cite

This paper surveys human-inspired speech technologies developed for European Portuguese and the computational models they integrate and made them possible. In this regard, it covers systems for synthesis and recognition as well as information on the methods adopted for the speech production studies that were performed, in parallel, to support them. And, on doing so, it can also contribute to provide an entry point for those who work in the field but are not familiar with these particular areas, including: context, history, and comprehensive references. As the great majority of work in these areas for European Portuguese was done by the first author’s research group, this paper can also be seen as a review of more than 25 years of research at University of Aveiro in these topics.

show abstract

A segmentation-informed deep learning framework to register dynamic two-dimensional magnetic resonance images of the vocal tract during speech

Cited by 3 publications

References 63 publications

Research in methodologies for modelling the oral cavity

Research in methodologies for modelling the oral cavity

Real-time speech MRI datasets with corresponding articulator ground-truth segmentations

Human-inspired computational models for European Portuguese: a review

Contact Info

Product

Resources

About