Automatic segmentation of speech articulators from real-time midsagittal MRI based on supervised learning

Labrunie, Mathieu; Badin, Pierre; Voit, Dirk; Joseph, Arun A.; Frahm, Jens; Lamalle, Laurent; Vilain, Coriandre; Boë, Louis‐Jean

doi:10.1016/j.specom.2018.02.004

Cited by 29 publications

(39 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If our hypothesis is correct, these two abilities should be strongly correlated, and cases where a species excels at identifying turn boundaries but performs poorly at identifying emotion should be rare or nonexistent. The question remains as to how perceptual sensitivity to temporal variations in vocalizations exchanged among individuals evolved into the ability to use temporal variations in the signal to predict unit boundaries in speech . We propose that the first step in this investigation is to analyze the effect of social factors (e.g., group size, group cohesion, and social complexity) on cross‐species ability for attentional tuning to temporal patterns in auditory signals.…”

Section: Discussionmentioning

confidence: 99%

“…The question remains as to how percep-tual sensitivity to temporal variations in vocalizations exchanged among individuals evolved into the ability to use temporal variations in the signal to predict unit boundaries in speech. 172 We propose that the first step in this investigation is to analyze the effect of social factors (e.g., group size, group cohesion, and social complexity) on cross-species ability for attentional tuning to temporal patterns in auditory signals. Building on the data reviewed here, we propose that the ability to predict turns in vocal interactions is positively correlated with the ability to predict unit boundaries in a vocal signal in animals.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Temporal modulation in speech, music, and animal vocal communication: evidence of conserved function

Filippi

Hoeschele

Spierings

et al. 2019

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

Speech is a distinctive feature of our species. It is the default channel for language and constitutes our primary mode of social communication. Determining the evolutionary origins of speech is a challenging prospect, in large part because it appears to be unique in the animal kingdom. However, direct comparisons between speech and other forms of acoustic communication, both in humans (music) and animals (vocalization), suggest that important components of speech are shared across domains and species. In this review, we focus on a single aspect of speech—temporal patterning—examining similarities and differences across speech, music, and animal vocalization. Additional structure is provided by focusing on three specific functions of temporal patterning across domains: (1) emotional expression, (2) social interaction, and (3) unit identification. We hypothesize an evolutionary trajectory wherein the ability to identify units within a continuous stream of vocal sounds derives from social vocal interaction, which, in turn, derives from vocal emotional communication. This hypothesis implies that unit identification has parallels in music and precursors in animal vocal communication. Accordingly, we demonstrate the potential of comparisons between fundamental domains of biological acoustic communication to provide insight into the evolution of language.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Temporal modulation in speech, music, and animal vocal communication: evidence of conserved function

Filippi

Hoeschele

Spierings

et al. 2019

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

show abstract

“…A related approach has been used by the University of Illinois group (the “partial separability” method; Fu et al, 2015), achieving a nominal rate of 100 fps for a single slice midsagittal scan and 25 fps for four simultaneously collected slices (see the method's application in Carignan, Shosted, Fu, Liang, & Sutton, 2015 and Johnson, Barlaz, Shosted, & Sutton, 2019). Another example is the “under‐sampled radial FLASH” method developed at Max‐Planck Institute, Göttingen (Niebergall et al, 2013; Frahm et al, 2014), which produces single‐slice scans at 55 fps (see the method's application in Labrunie et al, 2018). Overall, these developments are making rt‐MRI comparable in temporal resolution to EMA and surpassing most currently available ultrasound systems (see §2.2 in Part I).…”

Section: Entire Vocal Tractmentioning

confidence: 99%

“…The analysis of MRI data is rarely based on raw images; rather, these are post‐processed (e.g., re‐aligned to compensate for possible head movements) and segmented (traced). Generally, segmentation is done using two main methods (see Labrunie et al, 2018 for a review). One method involves identifying and tracing individual articulations.…”

Section: Entire Vocal Tractmentioning

confidence: 99%

Research methods in articulatory phoneticsII: Studying other gestures and recent trends

Kochetov

2020

Language and Linguist. Compass

View full text Add to dashboard Cite

This article is Part II of a general overview of current methods in articulatory phonetics research (which also consists of Part I "Research methods in articulatory phonetics I: Introduction & studying oral gestures"). The article begins by examining methods employed by phoneticians to investigate laryngeal and nasal gestures-speech-related configurations of the glottis and the lowering/raising the velum for nasal/oral consonants and vowels. This is done by reviewing the methods of electroglottography, endoscopy, photoglottography, and measurements of airflow and air pressure. The article further examines magnetic resonance imaging and radiography (X-rays)-the methods employed to investigate the entire vocal tract. The review is concluded with the methods of video recording and optical tracking, as used to study manual gestures in speech and sign language. Each methodological section contains a review of relevant journal publications illustrating the application of the method, as well as references to further readings. The article concludes with an overview of current developments and trends in instrumental articulatory phonetics, and highlight issues requiring further research.

show abstract

“…Articulatory speech production studies often rely on midsagittal images of the vocal tract area and Magnetic Resonance Imaging (MRI) constitutes in this approach an essential modality [8][9][10] . Identifying landmarks of the vocal tract area on these images has always been done manually or as a byproduct of manual segmentation 3,11 .…”

mentioning

confidence: 99%

Automatic vocal tract landmark localization from midsagittal MRI data

Eslami

Neuschaefer‐Rube

Serrurier

2020

Sci Rep

View full text Add to dashboard Cite

the various speech sounds of a language are obtained by varying the shape and position of the articulators surrounding the vocal tract. Analyzing their variations is crucial for understanding speech production, diagnosing speech disorders and planning therapy. identifying key anatomical landmarks of these structures on medical images is a pre-requisite for any quantitative analysis and the rising amount of data generated in the field calls for an automatic solution. The challenge lies in the high inter-and intra-speaker variability, the mutual interaction between the articulators and the moderate quality of the images. This study addresses this issue for the first time and tackles it by means of Deep Learning. it proposes a dedicated network architecture named Flat-net and its performance are evaluated and compared with eleven state-of-the-art methods from the literature. the dataset contains midsagittal anatomical Magnetic Resonance Images for 9 speakers sustaining 62 articulations with 21 annotated anatomical landmarks per image. Results show that the Flat-net approach outperforms the former methods, leading to an overall Root Mean Square Error of 3.6 pixels/0.36 cm obtained in a leave-oneout procedure over the speakers. the implementation codes are also shared publicly on GitHub.www.nature.com/scientificreports www.nature.com/scientificreports/ animals 22 . This short review emphasizes the importance of anatomical landmark localization from images for a large variety of applications and this study lengthens this non-exhaustive list with speech production.Within the existing literature, two fields of application appear more particularly active and connected to our problem. The first is the localization of landmarks for the face, rich of an abundant research from landmark identification on two-dimensional photographs 23,24 to landmark identification on three-dimensional Ultrasounds for fetuses for instance 25,26 . A recent and comprehensive review is provided by Wu et al. 27 . It represents a challenging issue due to the high variability of the shapes, poses, occlusions, and lighting conditions. Similarly, localizing the position of the joints of the body on images to estimate the human pose is also a long-standing problem 28 . It is also a challenging issue in computer vision due to the high variability of the postures, body shapes, actions, clothes and scenes.The goal and contribution of this study is to propose a fully-automated end-to-end image analysis methods for localizing key anatomical landmarks in the vocal tract area from midsagittal MRI data. As emphasized earlier, the need for such a method in the field is required and has never been attempted so far. It is aimed to be used in the future for new speakers for which no prior data are available. Considering the existing methods of the literature and the recent rise of data science to solve such problems, Deep Learning (DL) appears as the inescapable approach 29 . Indeed, DL approaches in image processing tasks appear to outperform most of traditional t...

show abstract

Automatic segmentation of speech articulators from real-time midsagittal MRI based on supervised learning

Cited by 29 publications

References 53 publications

Temporal modulation in speech, music, and animal vocal communication: evidence of conserved function

Temporal modulation in speech, music, and animal vocal communication: evidence of conserved function

Research methods in articulatory phoneticsII: Studying other gestures and recent trends

Automatic vocal tract landmark localization from midsagittal MRI data

Contact Info

Product

Resources

About