Speech Organ Contour Extraction Using Real-Time MRI and Machine Learning Method

Takemoto, Hironori; Goto, Tsubasa; Hagihara, Yuya; Hamanaka, Sayaka; Kitamura, Tatsuya; Maekawa, Kikuo

doi:10.21437/interspeech.2019-1593

Cited by 8 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous methodologies have implemented greater automation by leveraging powerful machine learning techniques. However, these make strong assumptions about the underlying data or rely on models which are trained from a narrow sample which reduces their generalisability to new research [12][13][14][15][16]. We have demonstrated that our pipeline generalises beyond the dataset for which it was designed.…”

Section: Discussionmentioning

confidence: 99%

“…Furthermore, those approaches which are available have not yet been demonstrated to generalise beyond the individual datasets for which they were developed. These methodologies typically involve automated or machine learning processes which are trained and tested against a narrow range of data, typically composed of a small number of speakers scanned at a single imaging centre [12][13][14][15][16][17][18]. The development of these techniques has sampled disproportionally from a single image repository [6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An open-source toolbox for measuring vocal tract shape from real-time magnetic resonance images

Belyk¹,

Carignan²,

McGettigan³

2021

Preprint

View full text Add to dashboard Cite

Real-time magnetic resonance imaging is a technique that provides high contrast videographic data of the vocal tract that allow researchers to observe the internal structures that shape the sounds of speech. However, structural features need to be extracted from these vocal tract images to make them useful to researchers. We have developed a semi-automated processing pipeline that produces outlines of the vocal tract to quantify vocal tract morphology. Our approach uses simple tissue classification constrained to pixels that analysts have identified as likely to contain the vocal tract and surrounding tissue. This approach is supplemented with multiple opportunities for the analyst to intervene in order to ensure that outputs are robust to errors. Although this approach is more labour intensive than more fully automated alternatives, these costs are offset by the benefits of improving the quality of measurements. We demonstrate that this pipeline can be generalised to a range of datasets and that it remains reliable across analysts, particularly among analysts with vocal tract expertise. The pipeline’s reliance on user input presents a challenge to scalability if applied to very large. Measurements produced by this pipeline could be provide a broader scope of training data for fully automated methods in an effort to improve their generalisability.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An open-source toolbox for measuring vocal tract shape from real-time magnetic resonance images

Belyk¹,

Carignan²,

McGettigan³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Dlib [34] is widely used in facial landmark localization tasks, where it is an implementation of Ensemble of Regression Trees (ERT) presented in [35]. A recent study successfully uses Dlib to detect vocal tract landmarks [21]. During the training of the Dlib, we observe that it is important to initialize the VT landmarks carefully.…”

Section: A Baseline Methodsmentioning

confidence: 99%

“…Recently, Takemoto et al [21] utilize the Dlib machine learning library [22] to estimate the VT contours of five speech organs. Their approach performs an accurate tracking of articulatory movements on unseen data using a few manually annotated images in the training set.…”

Section: Introductionmentioning

confidence: 99%

Vocal Tract Contour Tracking in rtMRI Using Deep Temporal Regression Network

Asadiabadi

Erzin

2020

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Recent advances in real-time Magnetic Resonance Imaging (rtMRI) provide an invaluable tool to study speech articulation. In this paper, we present an effective deep learning approach for supervised detection and tracking of vocal tract contours in a sequence of rtMRI frames. We train a single input multiple output deep temporal regression network (DTRN) to detect the vocal tract (VT) contour and the separation boundary between different articulators. The DTRN learns the non-linear mapping from an overlapping fixed-length sequence of rtMRI frames to the corresponding articulatory movements, where a blend of the overlapping contour estimates defines the detected VT contour. The detected contour is refined at a post-processing stage using an appearance model to further improve the accuracy of VT contour detection. The proposed VT contour tracking model is trained and evaluated over the USC-TIMIT dataset. Performance evaluation is carried out using three objective assessment metrics for the separating landmark detection, contour tracking and temporal stability of the contour landmarks in comparison with three baseline approaches from the recent literature. Results indicate significant improvements with the proposed method over the state-of-the-art baselines.

show abstract

“…Machine learning techniques were then developed with the advantage of learning from past examples, notably the "Active Shape Models" used by Labrunie et al [22]. Other machine learning techniques have been used [30], but in the last few years, deep learning, particularly Convolutional Neural Networks (CNNs), have brought significant progress in terms of performance. Initially proposed for tracking the tongue contour in ultrasound images with autoencoders [31], these techniques have been used for MRI images with the help of U-Net [32] as we did [21].…”

Section: Tracking Articulatorsmentioning

confidence: 99%