Detecting Nasal Vowels in Speech Interfaces Based on Surface Electromyography

Freitas, João; Teixeira, António; Silva, Samuel; Oliveira, Catarina; Dias, Miguel Sales

doi:10.1371/journal.pone.0127040

Cited by 10 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It can be seen in Figure 11 a,b that SSRNet provides excellent classification results for consonants and vowels. We observe the confusion between nasal and other consonants, which is consistent with [ 54 , 55 ]. This is due to the limitations of sEMG electrodes in detecting velum [ 55 ].…”

Section: Experiments and Resultssupporting

confidence: 89%

See 1 more Smart Citation

Sequence-to-Sequence Voice Reconstruction for Silent Speech in a Tonal Language

Wang

et al. 2022

Brain Sciences

View full text Add to dashboard Cite

Silent speech decoding (SSD), based on articulatory neuromuscular activities, has become a prevalent task of brain–computer interfaces (BCIs) in recent years. Many works have been devoted to decoding surface electromyography (sEMG) from articulatory neuromuscular activities. However, restoring silent speech in tonal languages such as Mandarin Chinese is still difficult. This paper proposes an optimized sequence-to-sequence (Seq2Seq) approach to synthesize voice from the sEMG-based silent speech. We extract duration information to regulate the sEMG-based silent speech using the audio length. Then, we provide a deep-learning model with an encoder–decoder structure and a state-of-the-art vocoder to generate the audio waveform. Experiments based on six Mandarin Chinese speakers demonstrate that the proposed model can successfully decode silent speech in Mandarin Chinese and achieve a character error rate (CER) of 6.41% on average with human evaluation.

show abstract

Section: Experiments and Resultssupporting

confidence: 89%

“…We observe the confusion between nasal and other consonants, which is consistent with [ 54 , 55 ]. This is due to the limitations of sEMG electrodes in detecting velum [ 55 ].…”

Section: Experiments and Resultssupporting

confidence: 89%

Sequence-to-Sequence Voice Reconstruction for Silent Speech in a Tonal Language

Wang

et al. 2022

Brain Sciences

View full text Add to dashboard Cite

show abstract

“…To our knowledge, no fMRI studies have examined the representation of velum control. Velum control includes the action of several/distributed muscles 56 , which limits its topographical localization. The results of the current study show that the classification of oral versus nasal sounds (i.e., velum control) using MVPA was significant within the pre-central gyri, bilaterally (Figs.…”

Section: Discussionmentioning

confidence: 99%

“…Taken together, our results suggest a more distributed representation of the velum control across M1. In future, a more deterministic understanding of the somatotopy of the velum may require investigating velar movements separately, for example by using continuous oral to nasal (and nasal to oral) vowel transitions, or by simultaneously obtaining the velum's position using real-time MRI 57,58 or non-invasive electromyogram recordings 56 .…”

Section: Discussionmentioning

confidence: 99%

Phonatory and articulatory representations of speech production in cortical and subcortical fMRI responses

Correia

Caballero-Gaudes

Guediche

et al. 2020

Sci Rep

View full text Add to dashboard Cite

Speaking involves coordination of multiple neuromotor systems, including respiration, phonation and articulation. Developing non-invasive imaging methods to study how the brain controls these systems is critical for understanding the neurobiology of speech production. Recent models and animal research suggest that regions beyond the primary motor cortex (M1) help orchestrate the neuromotor control needed for speaking, including cortical and sub-cortical regions. Using contrasts between speech conditions with controlled respiratory behavior, this fMRI study investigates articulatory gestures involving the tongue, lips and velum (i.e., alveolars versus bilabials, and nasals versus orals), and phonatory gestures (i.e., voiced versus whispered speech). Multivariate pattern analysis (MVPA) was used to decode articulatory gestures in M1, cerebellum and basal ganglia. Furthermore, apart from confirming the role of a mid-M1 region for phonation, we found that a dorsal M1 region, linked to respiratory control, showed significant differences for voiced compared to whispered speech despite matched lung volume observations. This region was also functionally connected to tongue and lip M1 seed regions, underlying its importance in the coordination of speech. Our study confirms and extends current knowledge regarding the neural mechanisms underlying neuromotor speech control, which hold promise to study neural dysfunctions involved in motor-speech disorders non-invasively. Despite scientific interest in verbal communication, the neural mechanisms supporting speech production remain unclear. The goal of the current study is to capture the underlying representations that support the complex orchestration of articulators, respiration, and phonation needed to produce intelligible speech. Importantly, voiced speech can be defined as an orchestrated task, where concerted phonation-articulation is mediated by respiration 1. In turn, a more detailed neural specification of these gestures in fluent speakers is necessary to develop biologically plausible models of speech production. The ability to image the speech production circuitry at work using non-invasive methods holds promise for future application in studies that aim to assess potential dysfunction. Upper motor-neurons located within the primary motor cortex (M1) exhibit a somatotopic organization that projects onto the brain-stem innervating the musculature of speech 2-6. This functional organization of M1 has been replicated with functional magnetic resonance imaging (fMRI) for the lip, tongue and jaw control regions 7-11. However, the articulatory control of the velum, which has an active role in natural speech (oral and nasal sounds) remains largely underspecified. Furthermore, laryngeal muscle control, critical for phonation, has more recently been mapped onto two separate areas in M1 4,5,12 : a ventral and a dorsal laryngeal motor area (vLMA and dLMA). Whereas the vLMA (ventral to the tongue motor area) is thought to operate the extrinsic laryngeal muscles, controlling the...

show abstract

“…The pitch problem in EL speech for Cantonese is solved using EMG (Shing et al 2016). Relationship between EMG and nasal vowels for Portuguese is established by real-time magnetic resonance imaging (Freitas et al 2015). German and English vowel is also recognized by root mean square signal and ANN (Arjunan et al 2006, 2007).…”

Section: Introductionmentioning

confidence: 99%

Voiceless Bangla vowel recognition using sEMG signal

et al. 2016

View full text Add to dashboard Cite

Some people cannot produce sound although their facial muscles work properly due to having problem in their vocal cords. Therefore, recognition of alphabets as well as sentences uttered by these voiceless people is a complex task. This paper proposes a novel method to solve this problem using non-invasive surface Electromyogram (sEMG). Firstly, eleven Bangla vowels are pronounced and sEMG signals are recorded at the same time. Different features are extracted and mRMR feature selection algorithm is then applied to select prominent feature subset from the large feature vector. After that, these prominent features subset is applied in the Artificial Neural Network for vowel classification. This novel Bangla vowel classification method can offer a significant contribution in voice synthesis as well as in speech communication. The result of this experiment shows an overall accuracy of 82.3 % with fewer features compared to other studies in different languages.

show abstract

Detecting Nasal Vowels in Speech Interfaces Based on Surface Electromyography

Cited by 10 publications

References 22 publications

Sequence-to-Sequence Voice Reconstruction for Silent Speech in a Tonal Language

Sequence-to-Sequence Voice Reconstruction for Silent Speech in a Tonal Language

Phonatory and articulatory representations of speech production in cortical and subcortical fMRI responses

Voiceless Bangla vowel recognition using sEMG signal

Contact Info

Product

Resources

About