Silent Speech Recognition as an Alternative Communication Device for Persons With Laryngectomy

Meltzner, Geoffrey S.; Heaton, James T.; Deng, Yunbin; Luca, Gianluca De; Roy, Serge H.; Kline, Joshua C.

doi:10.1109/taslp.2017.2740000

Cited by 92 publications

(62 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…31 It has been developed for silent articulation for laryngectomees and for subvocal speech allowing direct communication with computer devices. 32 It offers the most advanced algorithmic and hardware system, but with crucial design differences with our approach. Altec Inc. has developed custom active sEMG pairs (10 × 20 × 3 mm) incorporating sensing and conditioning electronics with double parallel bar electrodes spaced 1 cm apart for differential recording relative to a ground reference electrode on the lower cervical spine, communicating wirelessly to a processing base station.…”

Section: Discussionmentioning

confidence: 99%

Pilot study for a novel and personalized voice restoration device for patients with laryngectomy

Rameau¹

2019

Head & Neck

View full text Add to dashboard Cite

Background: The main modalities for voice restoration after laryngectomy are the electrolarynx, and the tracheoesophageal puncture [Correction added on 30 January 2020 after first online publication: The preceding sentence has been revised. It originally read "The main modalities for voice restoration after laryngectomy are the electrolarynx and the tracheoesophageal puncture."]. All have limitations and new technologies may offer innovative alternatives via silent speech. Objective: To describe a novel and personalized method of voice restoration using machine learning applied to electromyographic signal from articulatory muscles for the recognition of silent speech in a patient with total laryngectomy.Methods: Surface electromyographic (sEMG) signals of articulatory muscles were recorded from the face and neck of a patient with total laryngectomy who was articulating words silently. These sEMG signals were then used for automatic speech recognition via machine learning. Sensor placement was tailored to the patient's unique anatomy, following radiation and surgery. A personalized wearable mask covering the sensors was designed using 3D scanning and 3D printing.Results: Using seven sEMG sensors on the patient's face and neck and two grounding electrodes, we recorded EMG data while he was mouthing "Tedd" and "Ed." With data from 75 utterances for each of these words, we discriminated the sEMG signal with 86.4% accuracy using an XGBoost machinelearning model. Conclusions:This pilot study demonstrates the feasibility of sEMG-based alaryngeal speech recognition, using tailored sensor placement and a personalized wearable device. Further refinement of this approach could allow translation of silently articulated speech into a synthesized voiced speech via portable devices.

show abstract

Section: Discussionmentioning

confidence: 99%

Pilot study for a novel and personalized voice restoration device for patients with laryngectomy

Rameau¹

2019

Head & Neck

View full text Add to dashboard Cite

show abstract

“…In the second case, silent speech recognition (SSR) is applied on the biosignal which extracts the content spoken by the person (i.e., the result is text). This step is then followed by text-to-speech (TTS) synthesis [3], [4], [11]- [13], [15], [18], [24], [26]. A drawback of the SSR+TTS approach might be that the errors made by the SSR component inevitably appear as errors in the final TTS output [30], and also that it causes a significant end-to-end delay.…”

Section: Introductionmentioning

confidence: 99%

Autoencoder-Based Articulatory-to-Acoustic Mapping for Ultrasound Silent Speech Interfaces

Gosztolya

Pintér

Tóth

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

When using ultrasound video as input, Deep Neural Network-based Silent Speech Interfaces usually rely on the whole image to estimate the spectral parameters required for the speech synthesis step. Although this approach is quite straightforward, and it permits the synthesis of understandable speech, it has several disadvantages as well. Besides the inability to capture the relations between close regions (i.e. pixels) of the image, this pixelby-pixel representation of the image is also quite uneconomical. It is easy to see that a significant part of the image is irrelevant for the spectral parameter estimation task as the information stored by the neighbouring pixels is redundant, and the neural network is quite large due to the large number of input features. To resolve these issues, in this study we train an autoencoder neural network on the ultrasound image; the estimation of the spectral speech parameters is done by a second DNN, using the activations of the bottleneck layer of the autoencoder network as features. In our experiments, the proposed method proved to be more efficient than the standard approach: the measured normalized mean squared error scores were lower, while the correlation values were higher in each case. Based on the result of a listening test, the synthesized utterances also sounded more natural to native speakers. A further advantage of our proposed approach is that, due to the (relatively) small size of the bottleneck layer, we can utilize several consecutive ultrasound images during estimation without a significant increase in the network size, while significantly increasing the accuracy of parameter estimation.

show abstract

“…Meltzner, et. al [10]. This study focuses on patients who underwent surgical removal of the larynx due to trauma or diseases and thereby required an alternative voice source or assistive device to verbally communicate.…”

Section: Introductionmentioning

confidence: 99%

ECHO: Hands-Free Computer Interaction using Speech Recognition System for the Debilitated

Matel¹,

Aquines²,

Guzman³

et al. 2019

IJRTE

View full text Add to dashboard Cite

Computer nowadays is a must-have tool for most people. However, it is not a tool to be used by people with physical disabilities, especially the ones lacking an arm or two. The goal of this paper is to introduce a system that will help computer users perform tasks and make use of computer features and functions despite their physical limitations through the Speech Recognition System (SRS) in the English language. Ideally, this aims to provide users with an alternative way of interacting with the computer system and navigate through its functions using SRS in place of peripheral devices. It can be used to navigate through menus, open and manage applications, open certain websites, browse the internet, and type words, letters, numbers, and symbols using the dictation mode. For the testing phase, the following test cases were used: Functionality Testing, Stress Testing, and Compatibility. The testing phase yielded a result of 94.79% for the functional, 100% for stress, and 100% for compatibility, effectively ensuring that the software is working as intended. The evaluation results conforming to the standards of the ISO/IEC 9126-1 yielded a mean of 3.57 with a standard deviation of 0.52 interpreted as ‘Highly Acceptable', which means that the software can be used as an effective alternative to peripheral devices and can even be used to complement its usage.

show abstract

Silent Speech Recognition as an Alternative Communication Device for Persons With Laryngectomy

Cited by 92 publications

References 27 publications

Pilot study for a novel and personalized voice restoration device for patients with laryngectomy

Pilot study for a novel and personalized voice restoration device for patients with laryngectomy

Autoencoder-Based Articulatory-to-Acoustic Mapping for Ultrasound Silent Speech Interfaces

ECHO: Hands-Free Computer Interaction using Speech Recognition System for the Debilitated

Contact Info

Product

Resources

About